What should the recon-input be organized like if dwi images were not preprocessed using the qsiprep or UKB pipeline

Summary of what happened:

Hello,

I want to perform NoACT reconstruction on my diffusion data using qsiprep in recon-only mode. What should the recon-input be organized like? Note that my dwi images were not preprocessed using the qsiprep or UKB pipeline. I currently have preprocessed DWI images, corresponding .bval and .bvec files, and segmentation maps and masks extracted using iBEATs and registered to the subject’s DWI image space.

Command used (and if a helper script was used, a link to the helper script or the command generated):

PASTE CODE HERE

Version:

qsiprep v0.21.5.dev0+g36b93fe.d20240504

Environment (Docker, Singularity / Apptainer, custom installation):

Singularity

Data formatted according to a validatable standard? Please provide the output of the validator:

PASTE VALIDATOR OUTPUT HERE

Relevant log outputs (up to 20 lines):

PASTE LOG OUTPUT HERE

Screenshots / relevant information:


Hi @Shanbin_Zhang,

Here is an example layout:

bids/derivatives/qsiprep/
β”œβ”€β”€ dataset_description.json
β”œβ”€β”€ dwiqc.json # Not needed
β”œβ”€β”€ logs # Not needed
β”‚   β”œβ”€β”€ CITATION.html
β”‚   β”œβ”€β”€ CITATION.md
β”‚   └── CITATION.tex
β”œβ”€β”€ sub-01
β”‚   β”œβ”€β”€ anat
β”‚   β”‚   β”œβ”€β”€ sub-01_desc-aseg_dseg.nii.gz
β”‚   β”‚   β”œβ”€β”€ sub-01_desc-brain_mask.nii.gz
β”‚   β”‚   β”œβ”€β”€ sub-01_desc-preproc_T1w.nii
β”‚   β”‚   β”œβ”€β”€ sub-01_dseg.nii.gz
β”‚   β”‚   β”œβ”€β”€ sub-01_from-MNI152NLin2009cAsym_to-T1w_mode-image_xfm.h5
β”‚   β”‚   β”œβ”€β”€ sub-01_from-T1wACPC_to-T1wNative_mode-image_xfm.mat
β”‚   β”‚   β”œβ”€β”€ sub-01_from-T1wNative_to-T1wACPC_mode-image_xfm.mat
β”‚   β”‚   β”œβ”€β”€ sub-01_from-T1w_to-MNI152NLin2009cAsym_mode-image_xfm.h5
β”‚   β”‚   β”œβ”€β”€ sub-01_from-orig_to-T1w_mode-image_xfm.mat
β”‚   β”œβ”€β”€ dwi
β”‚   β”‚   β”œβ”€β”€ sub-01_confounds.tsv # not needed
β”‚   β”‚   β”œβ”€β”€ sub-01_desc-ImageQC_dwi.csv # not needed
β”‚   β”‚   β”œβ”€β”€ sub-01_desc-SliceQC_dwi.json # not needed
β”‚   β”‚   β”œβ”€β”€ sub-01_dwiqc.json # not needed
β”‚   β”‚   β”œβ”€β”€ sub-01_space-T1w_desc-brain_mask.nii.gz
β”‚   β”‚   β”œβ”€β”€ sub-01_space-T1w_desc-eddy_cnr.nii.gz # not needed
β”‚   β”‚   β”œβ”€β”€ sub-01_space-T1w_desc-preproc_dwi.b
β”‚   β”‚   β”œβ”€β”€ sub-01_space-T1w_desc-preproc_dwi.bval
β”‚   β”‚   β”œβ”€β”€ sub-01_space-T1w_desc-preproc_dwi.bvec
β”‚   β”‚   β”œβ”€β”€ sub-01_space-T1w_desc-preproc_dwi.nii.gz
β”‚   β”‚   └── sub-01_space-T1w_dwiref.nii.gz
β”‚   └── figures # not needed
β”‚       β”œβ”€β”€ sub-01_acq-98_dir-AP_dwi_denoise_acq_98_dir_AP_dwi_wf_denoising.svg
β”‚       β”œβ”€β”€ sub-01_acq-98_dir-AP_dwi_denoise_acq_98_dir_AP_dwi_wf_unringing.svg
β”‚       β”œβ”€β”€ sub-01_acq-98_dir-PA_dwi_denoise_acq_98_dir_PA_dwi_wf_denoising.svg
β”‚       β”œβ”€β”€ sub-01_acq-98_dir-PA_dwi_denoise_acq_98_dir_PA_dwi_wf_unringing.svg
β”‚       β”œβ”€β”€ sub-01_acq-99_dir-AP_dwi_denoise_acq_99_dir_AP_dwi_wf_denoising.svg
β”‚       β”œβ”€β”€ sub-01_acq-99_dir-AP_dwi_denoise_acq_99_dir_AP_dwi_wf_unringing.svg
β”‚       β”œβ”€β”€ sub-01_acq-99_dir-PA_dwi_denoise_acq_99_dir_PA_dwi_wf_denoising.svg
β”‚       β”œβ”€β”€ sub-01_acq-99_dir-PA_dwi_denoise_acq_99_dir_PA_dwi_wf_unringing.svg
β”‚       β”œβ”€β”€ sub-01_carpetplot.svg
β”‚       β”œβ”€β”€ sub-01_coreg.svg
β”‚       β”œβ”€β”€ sub-01_desc-resampled_b0ref.svg
β”‚       β”œβ”€β”€ sub-01_desc-sdc_b0t2w.svg
β”‚       β”œβ”€β”€ sub-01_desc-sdc_fa.svg
β”‚       β”œβ”€β”€ sub-01_final_denoise_wf_biascorr.svg
β”‚       β”œβ”€β”€ sub-01_sampling_scheme.gif
β”‚       β”œβ”€β”€ sub-01_seg_brainmask.svg
β”‚       └── sub-01_t1_2_mni.svg
└── sub-01.html # not needed

I will also note that NoACT pipelines will provide the worst tractograms, with ACT+hsvs pipelines performing the best. Unless there is a specific reason you need to use the no ACT pipeline, I would caution against it.

Best,
Steven

Hello, thank you very much for your answer and suggestions; they are very helpful to me. I will first do a test using the no ACT pipeline and later will use the ACT+hsvs pipelines for reconstruction. Thanks you for the example layout. I have two other little questions:

  1. I looked at the qsiprep source code, and it seems that qsiprep only recognizes and reconstructs DWI data marked as space-T1W during reconstruction. However, the current situation with my data is that T1w data (including segmentation maps) are registered to the DWI data. I hope to map the metric results to the anatomical space after fiber tracking and metric calculation, rather than registering the DWI data to the anatomical space before metric calculation. If I name the DWI data as space-T1W, will this cause any errors during reconstruction?
  2. I am currently processing a batch of DWI data from children aged 0-6 years. I want to modify the qsiprep pipeline to use iBEAT instead of FSL for structural segmentation and brain tissue extraction. Is this achievable?

Hi @Shanbin_Zhang

Qsiprep T1w space really means that DWI images are coregistered to the T1w image in ACPC space. I think as long as your images are registered to each other it will be fine.

I don’t know what iBEAT outputs look like, but the FSL tissue segmentation format isn’t too hard to match (just an image with different intensities per tissue type). The HSVS pipeline uses FreeSurfer instead, so it may not even be an issue. QSIprep also has an infant mode if you have any interest in reprocessing data.

Best,
Steven

1 Like

Hello, sorry to bother you again. I’d like to add a few details to my previous questions.

  • For the question about example layout, I’m not quite sure about the specific roles of the files in the anat modality folder during the reconstruction process. If I only want to perform a no-ACT reconstruction, is it correct that none of the files in the anat modality folder are needed? And if I need to do ACT reconstruction, do I only need to provide the segmentation map and mask in the DWI space, rather than all the files provided in the example? However, in practice, I found that if the anat modality files are not fully provided, an error occurs. Therefore, I had to use the structural preprocessing workflow of qsiprep to generate the necessary anat files. This also leads to some issues (see Question 2). Given my situation, would it be more appropriate to directly use mrtrix commands for the reconstruction?
  • Since the files in the dwi modality I provided in the recon_input folder are not preprocessed by qsiprep, but the anat modality files are generated by the structural preprocessing workflow of qsiprep, this causes a mismatch between the two spaces. In the reconstruction process, the resample_mask node produces an incorrect mask, leading to problems in the subsequent estimate_response node. Can I specify in qsiprep to directly use the sub-XX_space-T1w_desc-brain_mask.nii.gz from the dwi modality in the estimate_response node?
    I am a beginner student in the field of DWI, and my questions might be quite basic. Thank you again for your patience and assistance!

HI @Shanbin_Zhang,

No, that is not correct. The MNI transforms are used to warp the atlases to subject space. If you do not need the connectome parcellation, then maybe it will work if you modify the recon-spec to take out the atlases.

You will still need the transform files for the reason I stated above. If you use HSVS then you do not need a segmentation since it is calculated from FreeSurfer. Otherwise you will need the segmentation.

It would help to see what the error is.

That’s always an option and might be appropriate for you in this case.

No I don’t think so.

Best,
Steven

The error is

2:37:52,651 nipype.workflow INFO:
         [Node] Executing "sub_537715_ses_49_acq_multishell_space_T1w_desc_preproc_recon_inputs" <qsiprep.interfaces.interchange.ReconWorkflowInputs>
240715-22:37:52,662 nipype.workflow INFO:
         [Node] Finished "sub_537715_ses_49_acq_multishell_space_T1w_desc_preproc_recon_inputs", elapsed time 0.009352s.
240715-22:37:56,549 nipype.workflow INFO:
         [Node] Setting-up "qsirecon_wf.sub-11228_mrtrix_multishell_msmt_noACT.sub_11228_ses_09_acq_multishell_space_T1w_desc_preproc_recon_wf.msmt_csd.estimate_response" in "/work/qsirecon_wf/sub-11228_mrtrix_multishell_msmt_noACT/sub_11228_ses_09_acq_multishell_space_T1w_desc_preproc_recon_wf/msmt_csd/estimate_response".
240715-22:37:57,43 nipype.workflow INFO:
         [Node] Executing "estimate_response" <qsiprep.interfaces.mrtrix.SS3TDwi2Response>
240715-22:38:47,52 nipype.workflow INFO:
         [Node] Finished "estimate_response", elapsed time 49.802794s.
240715-22:38:47,57 nipype.workflow WARNING:
         [Node] Error on "qsirecon_wf.sub-11228_mrtrix_multishell_msmt_noACT.sub_11228_ses_09_acq_multishell_space_T1w_desc_preproc_recon_wf.msmt_csd.estimate_response" (/work/qsirecon_wf/sub-11228_mrtrix_multishell_msmt_noACT/sub_11228_ses_09_acq_multishell_space_T1w_desc_preproc_recon_wf/msmt_csd/estimate_response)
240715-22:38:48,435 nipype.workflow ERROR:
         Node estimate_response failed to run on host node-2-2.
240715-22:38:48,463 nipype.workflow ERROR:
         Saving crash info to /out/qsirecon/sub-11228/log/20240715-221403_54c90c4f-1279-4ce6-aa79-68db98b8bdff/crash-20240715-223848-sbzhang-estimate_response-80728fdb-61d7-42c3-8859-0c610e6013f2.txt
Traceback (most recent call last):
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node estimate_response.

Cmdline:
        dwi2response dhollander -mask /work/qsirecon_wf/sub-11228_mrtrix_multishell_msmt_noACT/sub_11228_ses_09_acq_multishell_space_T1w_desc_preproc_dwi_specific_anat_wf/resample_mask/sub-11228_desc-brain_mask_trans.nii.gz -nthreads 7 /work/qsirecon_wf/sub-11228_mrtrix_multishell_msmt_noACT/sub_11228_ses_09_acq_multishell_space_T1w_desc_preproc_recon_wf/msmt_csd/create_mif/sub-11228_ses-09_acq-multishell_space-T1w_desc-preproc_dwi.mif sub-11228_ses-09_acq-multishell_space-T1w_desc-preproc_dwi_wm.txt sub-11228_ses-09_acq-multishell_space-T1w_desc-preproc_dwi_gm.txt sub-11228_ses-09_acq-multishell_space-T1w_desc-preproc_dwi_csf.txt
Stdout:

Stderr:
        dwi2response: 
        dwi2response: Note that this script makes use of commands / algorithms that have relevant articles for citation. Please consult the help page (-help option) for more information.
        dwi2response: 
        dwi2response: Generated scratch directory: /work/qsirecon_wf/sub-11228_mrtrix_multishell_msmt_noACT/sub_11228_ses_09_acq_multishell_space_T1w_desc_preproc_recon_wf/msmt_csd/estimate_response/dwi2response-tmp-ILKU4J/
        dwi2response: Importing DWI data (/work/qsirecon_wf/sub-11228_mrtrix_multishell_msmt_noACT/sub_11228_ses_09_acq_multishell_space_T1w_desc_preproc_recon_wf/msmt_csd/create_mif/sub-11228_ses-09_acq-multishell_space-T1w_desc-preproc_dwi.mif)...
        dwi2response: Importing mask (/work/qsirecon_wf/sub-11228_mrtrix_multishell_msmt_noACT/sub_11228_ses_09_acq_multishell_space_T1w_desc_preproc_dwi_specific_anat_wf/resample_mask/sub-11228_desc-brain_mask_trans.nii.gz)...
        dwi2response: Changing to scratch directory (/work/qsirecon_wf/sub-11228_mrtrix_multishell_msmt_noACT/sub_11228_ses_09_acq_multishell_space_T1w_desc_preproc_recon_wf/msmt_csd/estimate_response/dwi2response-tmp-ILKU4J/)
        dwi2response: -------
        dwi2response: 7 unique b-value(s) detected: 5,499,1000,1500,2002,2500,3000 with 14,18,24,32,48,68,96 volumes
        dwi2response: -------
        dwi2response: Preparation:
        dwi2response: * Eroding brain mask by 3 pass(es)...
        dwi2response:   [ mask: 29758 -> 15977 ]
        dwi2response: * Computing signal decay metric (SDM):
        dwi2response:  * b=5...
        dwi2response:  * b=499...
        dwi2response:  * b=1000...
        dwi2response:  * b=1500...
        dwi2response:  * b=2002...
        dwi2response:  * b=2500...
        dwi2response:  * b=3000...
        dwi2response: * Removing erroneous voxels from mask and correcting SDM...
        dwi2response:   [ mask: 15977 -> 8372 ]
        dwi2response: -------
        dwi2response: Crude segmentation:
        dwi2response: * Crude WM versus GM-CSF separation (at FA=0.2)...
        dwi2response:   [ 8372 -> 8364 (WM) & 8 (GM-CSF) ]
        dwi2response: * Crude GM versus CSF separation...
        dwi2response:   [ 8 -> 7 (GM) & 1 (CSF) ]
        dwi2response: -------
        dwi2response: Refined segmentation:
        dwi2response: * Refining WM...
        dwi2response:   [ WM: 8364 -> 7941 ]
        dwi2response: * Refining GM...
        dwi2response:   [ GM: 7 -> 4 ]
        dwi2response: * Refining CSF...
        dwi2response:   [ CSF: 1 -> 11 ]
        dwi2response: -------
        dwi2response: Final voxel selection and response function estimation:
        dwi2response: * CSF:
        dwi2response:  * Selecting final voxels (10.0% of refined CSF)...
        dwi2response:    [ CSF: 11 -> 1 ]
        dwi2response:  * Estimating response function...
        dwi2response: * GM:
        dwi2response:  * Selecting final voxels (2.0% of refined GM)...

        dwi2response: [ERROR] mrcalc refined_gm.mif safe_sdm.mif 0.104317948 -subtract -abs 1 -add 0 -if - | mrthreshold - - -bottom 0 -ignorezero | mrcalc refined_gm.mif - 0 -if - -datatype bit | mrconvert - voxels_gm.mif -axes 0,1,2 (dhollander.py:228)
        dwi2response: [ERROR] Information from failed command:
        dwi2response:
                      mrcalc: computing: (refined_gm.mif ? (|(safe_sdm.mif - 0.104318)| + 1) : 0)... [==================================================]
                      mrthreshold: [ERROR] value supplied for option "bottom" is out of bounds (valid range: 1 to 9223372036854775807, value supplied: 0)
                      mrcalc: [ERROR] Could not interpret string "-" as either an image path or a numerical value
                      mrcalc: [ERROR] As image: 
                      mrcalc: [ERROR] no filename supplied to standard input (broken pipe?)
                      mrcalc: [ERROR] error opening image "-"
                      mrcalc: [ERROR] As numerical value: 
                      mrcalc: [ERROR] error converting string "-" to complex float (no valid conversion)
                      mrconvert: [ERROR] no filename supplied to standard input (broken pipe?)
                      mrconvert: [ERROR] error opening image "-"
        dwi2response:
        dwi2response: [ERROR] For debugging, inspect contents of scratch directory: /work/qsirecon_wf/sub-11228_mrtrix_multishell_msmt_noACT/sub_11228_ses_09_acq_multishell_space_T1w_desc_preproc_recon_wf/msmt_csd/estimate_response/dwi2response-tmp-ILKU4J/
        dwi2response: Scratch directory retained; location: /work/qsirecon_wf/sub-11228_mrtrix_multishell_msmt_noACT/sub_11228_ses_09_acq_multishell_space_T1w_desc_preproc_recon_wf/msmt_csd/estimate_response/dwi2response-tmp-ILKU4J/
Traceback:
        RuntimeError: subprocess exited with code 1.

While I check the output sub-11228_desc-brain_mask_trans.nii.gz of resample_mask node, which execute the command:

antsApplyTransforms --default-value 0 --dimensionality 3 --float 0 --input /out/qsiprep/sub-11228/anat/sub-11228_desc-brain_mask.nii.gz --interpolation NearestNeighbor --output sub-11228_desc-brain_mask_trans.nii.gz --reference-image /out/qsiprep/sub-11228/ses-09/dwi/sub-11228_ses-09_acq-multishell_space-T1w_dwiref.nii.gz --transform identity

And I found the output is incorrect because actually the input mask and the reference_image are not in the same space.

Actually I do not need to compute the connectome parcellation for now.

Hi @Shanbin_Zhang,

That error looks like a file could not be found. I do not know what your file organization looks like so I cannot say for sure what is missing.

The DWI and T1w (and all related derivatives) should be aligned to one another.

Best,
Steven

1 Like

Thank you very much!