How to add custom altas in QSIRecon:1.0.0

Summary of what happened:

In the QSIRecon documentation, I saw that if using a custom atlas, you need to specify --atlases and --datasets , but I received an error saying atlas_configs was not found.

Besides, the MNI152NLin2009cAsym documentation mentioned report 404 not found. I can’t transform my atlas space. Is the transformation space step not nessecery anymore?

Command used (and if a helper script was used, a link to the helper script or the command generated):

#!/bin/bash
# ... some code unrelated to qsirecon

sub=$FILE
fs_license=/opt/software/freesurfer/license.txt
input_dir=/data/disk0/zhangyi/LGStroke/prep/Left/prep_result/$sub/qsiprep
output_dir=/data/disk0/zhangyi/LGStroke/prep/Left/prep_result/$sub/qsirecon
work_dir=/data/disk0/zhangyi/LGStroke/prep/Left/prep_result/$sub/qsirecon_work
cores=8
pipeline=mrtrix_singleshell_ss3t_ACT-hsvs
atlas_dir=/data/disk0/zhangyi/LGStroke/prep/Left/prep_result/atlases
atlas=Schaefer400Yeo2011networks17
fs_dir=/data/disk0/zhangyi/LGStroke/prep/Left/prep_result/$sub/freesurfer
mkdir -p $work_dir
mkdir -p $output_dir
docker run \
       --rm \
       --user $(id -u):$(id -g) \
	   -v $fs_license:/opt/freesurfer/license.txt:ro \
	   -v $input_dir:/input:ro \
	   -v $output_dir:/output \
	   -v $atlas_dir:/atlases \
	   -v $work_dir:/work \
	   -v $fs_dir:/fs_dir:ro \
       gong.wg:5000/qsirecon:latest \
       /input \
       /output \
       participant \
       --work-dir /work \
       --nthreads $cores \
       --omp-nthreads $cores \
       --recon-spec $pipeline \
       --datasets /atlases \
       --atlases $atlas \
       --fs-subjects-dir /fs_dir \
       --participant-label $sub

Version:

pennlinc/qsirecon:1.0.0

Relevant log outputs (up to 20 lines):

Traceback (most recent call last):
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py", line 66, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 525, in run
    result = self._run_interface(execute=True)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 643, in _run_interface
    return self._run_command(execute)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 720, in _run_command
    result = self._interface.run(cwd=outdir, ignore_exception=True)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 392, in run
    self._check_mandatory_inputs()
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 277, in _check_mandatory_inputs
    raise ValueError(msg)
ValueError: MRTrixAtlasGraph requires a value for input 'atlas_configs'. For a list of required inputs, see MRTrixAtlasGraph.help()

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/envs/qsiprep/lib/python3.10/concurrent/futures/process.py", line 246, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py", line 69, in run_node
    result["result"] = node.result
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 221, in result
    return _load_resultfile(
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/utils.py", line 290, in load_resultfile
    raise FileNotFoundError(results_file)
FileNotFoundError: /work/qsirecon_1_0_wf/sub-L0011_mrtrix_singleshell_ss3_hsvst/sub_L0011_ses_1_space_T1w_desc_preproc_recon_wf/mrtrix_conn/calc_connectivity/result_calc_connectivity.pklz

Screenshots / relevant information:

zhangyi@node4:/data/disk0/zhangyi/LGStroke/prep/Left/prep_result$ tree -L 2 atlases
atlases/
├── atlas-Schaefer400Yeo2011networks17
│   ├── atlas-Schaefer400Yeo2011networks17_res-1_space-FSLMNI152.nii.gz
│   └── atlas-Schaefer400Yeo2011networks17_res-1_space-FSLMNI152.tsv
└── dataset_description.json

Sincere thanks

Hi @watcher_man,

Please try naming it:

tpl-MNI152NLin6Asym_atlas-Schaefer400Yeo2011networks17_dseg.json <-- you should add that file
tpl-MNI152NLin6Asym_atlas-Schaefer400Yeo2011networks17_res-01_dseg.nii.gz

and

atlas-Schaefer400Yeo2011networks17_dseg.tsv

Although keep in mind, you should instead be using the MNI152NLin2009cAsym space. You can use the transformations that are available in templateflow: GitHub - templateflow/tpl-MNI152NLin2009cAsym at e797e29df0fbfc87991c93a03369035897e2b513

Also, why not use the Schaefer 400 atlas already in qsirecon?

Best,
Steven

Thank you very much for your help!

I need to use a specific version of the Schaefer400 atlas, downloaded from GitHub Yan2023_Schaefer2023, but it is in FSLMNI152 space, which seems to be different from the MNI152NLin2009cAsym mentioned in the documentation?
So according to your advice, it seems I still need to re-register?

Hi @watcher_man,

Yes I believe you need to register it. The FSL MNI space is MNI152NLin6Asym.

Best,
Steven

It’s really sad, I still got the same error message

exception calling callback for <Future at 0x7163d7c437f0 state=finished raised FileNotFoundError>
concurrent.futures.process._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py", line 66, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 525, in run
    result = self._run_interface(execute=True)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 643, in _run_interface
    return self._run_command(execute)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 720, in _run_command
    result = self._interface.run(cwd=outdir, ignore_exception=True)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 392, in run
    self._check_mandatory_inputs()
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 277, in _check_mandatory_inputs
    raise ValueError(msg)
ValueError: MRTrixAtlasGraph requires a value for input 'atlas_configs'. For a list of required inputs, see MRTrixAtlasGraph.help()

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/envs/qsiprep/lib/python3.10/concurrent/futures/process.py", line 246, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py", line 69, in run_node
    result["result"] = node.result
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 221, in result
    return _load_resultfile(
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/utils.py", line 290, in load_resultfile
    raise FileNotFoundError(results_file)
FileNotFoundError: /work/qsirecon_1_0_wf/sub-L0011_mrtrix_singleshell_ss3_hsvst/sub_L0011_ses_1_space_T1w_desc_preproc_recon_wf/mrtrix_conn/calc_connectivity/result_calc_connectivity.pklz
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/envs/qsiprep/lib/python3.10/concurrent/futures/_base.py", line 342, in _invoke_callbacks
    callback(self)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py", line 158, in _async_callback
    result = args.result()
  File "/opt/conda/envs/qsiprep/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/opt/conda/envs/qsiprep/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
FileNotFoundError: /work/qsirecon_1_0_wf/sub-L0011_mrtrix_singleshell_ss3_hsvst/sub_L0011_ses_1_space_T1w_desc_preproc_recon_wf/mrtrix_conn/calc_connectivity/result_calc_connectivity.pklz

This is my atlases folder

zhangyi@node4:/data/disk0/zhangyi/LGStroke/prep/Left/prep_result/atlases$ tree 
.
├── atlas-Schaefer400Yeo2011networks17
│   ├── atlas-Schaefer400Yeo2011networks17_dseg.tsv
│   ├── tpl-MNI152NLin2009cAsym_atlas-Schaefer400Yeo2011networks17_dseg.json
│   └── tpl-MNI152NLin2009cAsym_atlas-Schaefer400Yeo2011networks17_res-01_dseg.nii.gz
└── dataset_description.json

Attach docker information

pennlinc/qsirecon             latest    2e1cae5699a3   9 days ago      15.4GB

Attempting to submit the content of the sh file to slurm

sub=$FILE
fs_license=/opt/software/freesurfer/license.txt
input_dir=/data/disk0/zhangyi/LGStroke/prep/Left/prep_result/$sub/qsiprep
echo input_dir $input_dir
output_dir=/data/disk0/zhangyi/LGStroke/prep/Left/prep_result/$sub/qsirecon
echo output_dir $output_dir
work_dir=/data/disk0/zhangyi/LGStroke/prep/Left/prep_result/$sub/qsirecon_work
echo work_dir $work_dir
cores=8
pipeline=mrtrix_singleshell_ss3t_ACT-hsvs
atlas_dir=/data/disk0/zhangyi/LGStroke/prep/Left/prep_result/atlases
atlas=Schaefer400Yeo2011networks17
fs_dir=/data/disk0/zhangyi/LGStroke/prep/Left/prep_result/$sub/freesurfer
echo fs_dir $fs_dir
#rm -rf $work_dir $output_dir
mkdir -p $work_dir
mkdir -p $output_dir
docker run \
       --rm \
       --user $(id -u):$(id -g) \
	   -v $fs_license:/opt/freesurfer/license.txt:ro \
	   -v $input_dir:/input:ro \
	   -v $output_dir:/output \
	   -v $atlas_dir:/atlases \
	   -v $work_dir:/work \
	   -v $fs_dir:/fs_dir:ro \
       pennlinc/qsirecon:latest \
       /input \
       /output \
       participant \
       --work-dir /work \
       --nthreads $cores \
       --omp-nthreads $cores \
       --recon-spec $pipeline \
       --datasets /atlases \
       --atlases $atlas \
       --fs-subjects-dir /fs_dir \
       --participant-label $sub

the log file contains

zhangyi@node4:/data/disk0/zhangyi/LGStroke/prep/Left/log$ tail QSIRecon_123080_0.out 
250123-14:48:30,51 nipype.workflow INFO:
	 [Node] Executing "tck_sift2" <qsirecon.interfaces.mrtrix.SIFT2>
250123-15:01:25,404 nipype.workflow INFO:
	 [Node] Finished "tck_sift2", elapsed time 775.351797s.
250123-15:10:22,140 nipype.workflow INFO:
	 [Node] Setting-up "qsirecon_1_0_wf.sub-L0011_mrtrix_singleshell_ss3_hsvst.sub_L0011_ses_1_space_T1w_desc_preproc_recon_wf.mrtrix_conn.calc_connectivity" in "/work/qsirecon_1_0_wf/sub-L0011_mrtrix_singleshell_ss3_hsvst/sub_L0011_ses_1_space_T1w_desc_preproc_recon_wf/mrtrix_conn/calc_connectivity".
250123-15:10:22,153 nipype.workflow INFO:
	 [Node] Executing "calc_connectivity" <qsirecon.interfaces.mrtrix.MRTrixAtlasGraph>
250123-15:10:22,153 nipype.workflow WARNING:
	 [Node] Error on "qsirecon_1_0_wf.sub-L0011_mrtrix_singleshell_ss3_hsvst.sub_L0011_ses_1_space_T1w_desc_preproc_recon_wf.mrtrix_conn.calc_connectivity" (/work/qsirecon_1_0_wf/sub-L0011_mrtrix_singleshell_ss3_hsvst/sub_L0011_ses_1_space_T1w_desc_preproc_recon_wf/mrtrix_conn/calc_connectivity)

I don’t know where the problem is

Hi @watcher_man,

Can you try mounting your atlas directly to the qsirecon atlas folder /atlas/qsirecon_atlases/, that is, adding to your docker run arguments:

-v /path/to/your/tpl-MNI152NLin2009cAsym_atlas-Schaefer400Yeo2011networks17_res-01_dseg.nii.gz:/atlas/qsirecon_atlases/tpl-MNI152NLin2009cAsym_res-01_atlas-Schaefer400Yeo2011networks17_desc-400Parcels17NetworksLPS_dseg.nii.gz

and temporarily removing the --datasets /atlases line?

Best,
Steven

QAQ

not work

atlas_dir=/data/disk0/zhangyi/LGStroke/prep/Left/prep_result/atlases/atlas-Schaefer400Yeo2011networks17
atlas=Schaefer400Yeo2011networks17

docker run \
       --rm \
       --user $(id -u):$(id -g) \
	   -v $fs_license:/opt/freesurfer/license.txt:ro \
	   -v $input_dir:/input:ro \
	   -v $output_dir:/output \
	   -v $atlas_dir:/atlas/qsirecon_atlases/atlas-Schaefer400Yeo2011networks17 \
	   -v $work_dir:/work \
	   -v $fs_dir:/fs_dir:ro \
       pennlinc/qsirecon:latest \
       /input \
       /output \
       participant \
       --work-dir /work \
       --nthreads $cores \
       --omp-nthreads $cores \
       --recon-spec $pipeline \
       --atlases $atlas \
       --fs-subjects-dir /fs_dir \
       --participant-label $sub

Hi @watcher_man,

Can you try the exact code I supplied you (but replace /path/to/your with your actual path)?

Best,
Steven

Still the same error. QAQ

atlas_dir=/data/disk0/zhangyi/LGStroke/prep/Left/prep_result/atlases/atlas-Schaefer400Yeo2011networks17
atlas=Schaefer400Yeo2011networks17

	   -v $atlas_dir/tpl-MNI152NLin2009cAsym_atlas-Schaefer400Yeo2011networks17_res-01_dseg.nii.gz:/atlas/qsirecon_atlases/tpl-MNI152NLin2009cAsym_res-01_atlas-Schaefer400Yeo2011networks17_desc-400Parcels17NetworksLPS_dseg.nii.gz \

I think you need your atlases to be organized in the following manner:

├── atlas-Schaefer400Yeo2011networks17
│   ├── atlas-Schaefer400Yeo2011networks17_dseg.tsv
│   ├── atlas-Schaefer400Yeo2011networks17_space-MNI152NLin2009cAsym_dseg.json
│   └── atlas-Schaefer400Yeo2011networks17_space-MNI152NLin2009cAsym_res-01_dseg.nii.gz

As @smeisler noted, they need to be in MNI152NLin2009cAsym space. QSIRecon only supports the MNI152NLin2009cAsym and MNIInfant spaces right now.

You should mount the atlas dataset rather than the atlas’s specific folder, so your original mount point was correct as far as I can tell. I think the only problem is the space and naming convention.

QSIRecon should probably raise a more informative error when the atlases aren’t found.

I got a new error

ValueError: 'label' column not found in /atlas/qsirecon_atlases/atlas-Schaefer400Yeo2011networks17/atlas-Schaefer400Yeo2011networks17_dseg.tsv

It looks like QSIRecon recognized my atlas, thank you all very much!!!