Bug in QSIPREP reconstruction pipeline when using --dwi-only

1

Hi,

I am trying to use the qsiprep reconstruction pipeline but the execution quickly crashes, I assume I have a mistake in the command I evoke:

  • QSIPREP version: 0.15.2
  • singularity installation
  • freesurfer folder was created by a previous fmriprep (v 21.0.2) execution
  • command :
study=EcriPark
EXPDIR=/scratch/jsein/BIDS/$study
OUTDIR=
list_sub='CTR13'
part=participant  

singularity run --cleanenv -B /scratch/jsein/BIDS:/work \
 --nv /scratch/jsein/my_images/qsiprep-0.15.2.sif /work/$study  \
 /work/$study/derivatives/$OUTDIR $part --participant_label $list_sub     \
 -w /work/temp_data_${study}  --output-resolution 1.8  --fs-license-file /work/freesurfer/license.txt \
--template MNI152NLin2009cAsym \
 --recon-only --recon-spec mrtrix_multishell_msmt_ACT-hsvs --recon-input /work/$study/derivatives/qsiprep \
--freesurfer-input /work/$study/derivatives/fmriprep/sourcedata/freesurfer

Error output:

Process Process-2:
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/interfaces/base/traits_extension.py", line 129, in validate
    value = Path(value)  # Use pathlib's validation
  File "/usr/local/miniconda/lib/python3.8/pathlib.py", line 1042, in __new__
    self = cls._from_parts(args, init=False)
  File "/usr/local/miniconda/lib/python3.8/pathlib.py", line 683, in _from_parts
    drv, root, parts = self._parse_args(args)
  File "/usr/local/miniconda/lib/python3.8/pathlib.py", line 667, in _parse_args
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/local/miniconda/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/cli/run.py", line 1169, in build_recon_workflow
    retval['workflow'] = init_qsirecon_wf(
  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/workflows/recon/base.py", line 89, in init_qsirecon_wf
    single_subject_wf = init_single_subject_wf(
  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/workflows/recon/base.py", line 234, in init_single_subject_wf
    dwi_recon_wf = init_dwi_recon_workflow(
  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/workflows/recon/build_workflow.py", line 48, in init_dwi_recon_workflow
    registered_anat_wf, available_anatomical_data = init_dwi_recon_anatomical_workflow(
  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/workflows/recon/anatomical.py", line 432, in init_dwi_recon_anatomical_workflow
    nio.FreeSurferSource(
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 202, in __init__
    self.inputs.trait_set(**inputs)
  File "/usr/local/miniconda/lib/python3.8/site-packages/traits/has_traits.py", line 1520, in trait_set
    setattr(self, name, value)
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/interfaces/base/traits_extension.py", line 131, in validate
    self.error(objekt, name, str(value))
  File "/usr/local/miniconda/lib/python3.8/site-packages/traits/base_trait_handler.py", line 74, in error
    raise TraitError(
traits.trait_errors.TraitError: The 'subjects_dir' trait of a FSSourceInputSpec instance must be a pathlike object or string representing an existing directory, but a value of 'None' <class 'str'> was specified.

Output:

220612-00:57:58,46 nipype.utils WARNING:
	 A newer version (1.8.1) of nipy/nipype is available. You are using 1.7.0
220612-00:58:40,69 nipype.workflow IMPORTANT:
	 
    Running qsirecon version 0.15.2:
      * BIDS dataset path: /work/EcriPark.
      * Participant list: ['CTR13'].
      * Run identifier: 20220612-005758_b847161d-1829-4d59-9527-a8bf7dc2e845.
    
220612-00:58:40,421 nipype.workflow INFO:
	 found ['/work/EcriPark/derivatives/qsiprep/sub-CTR13/dwi/sub-CTR13_space-T1w_desc-preproc_dwi.nii.gz'] in /work/EcriPark/derivatives/qsiprep
220612-00:58:40,423 nipype.workflow INFO:
	 Missing T1w QSIPrep outputs found: /work/EcriPark/derivatives/qsiprep/sub-CTR13/anat/sub-CTR13_desc-brain_mask.nii.gz /work/EcriPark/derivatives/qsiprep/sub-CTR13/anat/sub-CTR13_desc-preproc_T1w.nii.gz
220612-00:58:40,423 nipype.workflow INFO:
	 Missing T1w QSIPrep outputs found: /work/EcriPark/derivatives/qsiprep/sub-CTR13/anat/sub-CTR13_from-T1w_to-MNI152NLin2009cAsym_mode-image_xfm.h5 /work/EcriPark/derivatives/qsiprep/sub-CTR13/anat/sub-CTR13_from-MNI152NLin2009cAsym_to-T1w_mode-image_xfm.h5
220612-00:58:40,424 nipype.workflow INFO:
	 Freesurfer directory /work/EcriPark/derivatives/fmriprep/sourcedata/freesurfer/sub-CTR13 exists for CTR13
220612-00:58:40,426 nipype.workflow INFO:
	 FreeSurfer data will be used to create a HSVS 5tt image.
220612-00:58:40,427 nipype.workflow INFO:
	 Anatomical (T1w) available for recon: {'has_qsiprep_5tt_fast': False, 'has_qsiprep_5tt_hsvs': False, 'has_freesurfer_5tt_hsvs': True, 'has_freesurfer': True, 'has_qsiprep_t1w': False, 'has_qsiprep_t1w_transforms': False}

Can somebody point to what I am doing wrong?

Thank you!

2

Hmm, that looks okay from the surface, except you can remove --template MNI152NLin2009cAsym, which is a deprecated flag.

Can you singularity shell in to the container, binding the same drives, and navigate to /work/$study/derivatives/fmriprep/sourcedata/freesurfer?

Best,
Steven

1 Like
3

Hi Steven,

Thank you for your insight!
When I do a singularity shell while binding the same drives, the variable $study within the singularity is not recognized anymore within the container shell but I can however navigate to the mounted drive: /work/EcriPark/derivatives/fmriprep/sourcedata/freesurfer

For sanity I tried the same command without using the variable $study but I still get the same error.

singularity run --cleanenv -B /scratch/jsein/BIDS:/work \
 --nv /scratch/jsein/my_images/qsiprep-0.15.2.sif /work/EcriPark  \
 /work/EcriPark/derivatives $part --participant_label $list_sub     \
 -w /work/temp_data_EcriPark  --fs-license-file /work/freesurfer/license.txt \
 --recon-only --recon-spec mrtrix_multishell_msmt_ACT-hsvs --recon-input /work/EcriPark/derivatives/qsiprep \
--freesurfer-input /work/EcriPark/derivatives/fmriprep/sourcedata/freesurfer

error:

traits.trait_errors.TraitError: The 'subjects_dir' trait of a FSSourceInputSpec instance must be a pathlike object or string representing an existing directory, but a value of 'None' <class 'str'> was specified.

Any idea? Thank you!

4

Hmm, thanks for the check. It makes sense $study isn’t carried into the container, since it’s a local variable, but it doesn’t make sense to me that the command is not working given the folder appears to be mounted correctly. Can you try doing this on v0.16.0RC3? 0.15.2 is actually deprecated for connectivity analyses due to incorrect atlas warping. If the error persists, I think the next step would be to open an issue on github.

Best,
Steven

5

Indeed, I just checked the GitHub page about the version 0.16.0RC3, it sounds pretty serious:

MAJOR UPDATES AND BUGFIXES: We do not recommend using 0.15 for reconstruction workflows.

I am giving it a go and will let you know how it goes.

Thank you for all the knowledge you share on Neurostars and elsewhere!

6

Hum, I just checked with v0.16.0RC3 but got the same error. I will proceed by opening an issue on GitHub.

error log:

WARNING:py.warnings:TemplateFlow: repository not found at /home/qsiprep/.cache/templateflow. Populating a new TemplateFlow stub.
If the path reported above is not the desired location for TemplateFlow, please set the TEMPLATEFLOW_HOME environment variable.
WARNING:py.warnings:TemplateFlow: repository not found at /home/qsiprep/.cache/templateflow. Populating a new TemplateFlow stub.
If the path reported above is not the desired location for TemplateFlow, please set the TEMPLATEFLOW_HOME environment variable.
IMPORTANT:nipype.workflow:
    Running qsirecon version 0.16.0RC3:
      * BIDS dataset path: /work/EcriPark.
      * Participant list: ['CTR13'].
      * Run identifier: 20220614-235215_7832af95-a925-4521-87dd-2badf42b1f9c.
    
INFO:nipype.workflow:found ['/work/EcriPark/derivatives/qsiprep/sub-CTR13/dwi/sub-CTR13_space-T1w_desc-preproc_dwi.nii.gz'] in /work/EcriPark/derivatives/qsiprep
INFO:nipype.workflow:Missing T1w QSIPrep outputs found: /work/EcriPark/derivatives/qsiprep/sub-CTR13/anat/sub-CTR13_desc-brain_mask.nii.gz /work/EcriPark/derivatives/qsiprep/sub-CTR13/anat/sub-CTR13_desc-preproc_T1w.nii.gz
INFO:nipype.workflow:Missing T1w QSIPrep outputs found: /work/EcriPark/derivatives/qsiprep/sub-CTR13/anat/sub-CTR13_from-T1w_to-MNI152NLin2009cAsym_mode-image_xfm.h5 /work/EcriPark/derivatives/qsiprep/sub-CTR13/anat/sub-CTR13_from-MNI152NLin2009cAsym_to-T1w_mode-image_xfm.h5
INFO:nipype.workflow:Freesurfer directory /work/EcriPark/derivatives/fmriprep/sourcedata/freesurfer/sub-CTR13 exists for CTR13
INFO:nipype.workflow:FreeSurfer data will be used to create a HSVS 5tt image.
INFO:nipype.workflow:Anatomical (T1w) available for recon: {'has_qsiprep_5tt_fast': False, 'has_qsiprep_5tt_hsvs': False, 'has_freesurfer_5tt_hsvs': True, 'has_freesurfer': True, 'has_qsiprep_t1w': False, 'has_qsiprep_t1w_transforms': False}
Process Process-2:
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/interfaces/base/traits_extension.py", line 129, in validate
    value = Path(value)  # Use pathlib's validation
  File "/usr/local/miniconda/lib/python3.8/pathlib.py", line 1042, in __new__
    self = cls._from_parts(args, init=False)
  File "/usr/local/miniconda/lib/python3.8/pathlib.py", line 683, in _from_parts
    drv, root, parts = self._parse_args(args)
  File "/usr/local/miniconda/lib/python3.8/pathlib.py", line 667, in _parse_args
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/local/miniconda/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/cli/run.py", line 1169, in build_recon_workflow
    retval['workflow'] = init_qsirecon_wf(
  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/workflows/recon/base.py", line 89, in init_qsirecon_wf
    single_subject_wf = init_single_subject_wf(
  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/workflows/recon/base.py", line 232, in init_single_subject_wf
    dwi_recon_wfs[dwi_file] = init_dwi_recon_workflow(
  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/workflows/recon/build_workflow.py", line 49, in init_dwi_recon_workflow
    registered_anat_wf, available_anatomical_data = init_dwi_recon_anatomical_workflow(
  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/workflows/recon/anatomical.py", line 440, in init_dwi_recon_anatomical_workflow
    nio.FreeSurferSource(
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 202, in __init__
    self.inputs.trait_set(**inputs)
  File "/usr/local/miniconda/lib/python3.8/site-packages/traits/has_traits.py", line 1520, in trait_set
    setattr(self, name, value)
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/interfaces/base/traits_extension.py", line 131, in validate
    self.error(objekt, name, str(value))
  File "/usr/local/miniconda/lib/python3.8/site-packages/traits/base_trait_handler.py", line 74, in error
    raise TraitError(
traits.trait_errors.TraitError: The 'subjects_dir' trait of a FSSourceInputSpec instance must be a pathlike object or string representing an existing directory, but a value of 'None' <class 'str'> was specified.

Thanks!

Julien

7

Good luck! at least it seems like the FS inputs are recognized according to the beginning of the code.

8

The discussion will hopefully go on on this issue thread:

9

Shot in the dark, but it may have to do with the -e / --cleanenv singularity flag (even though its the right thing to do in theory). Do you have FreeSurfer installed and in your $PATH on your machine? If so, can you try running the same command without that flag? I wonder is the container is just missing some relevant FreeSurfer information, and perhaps bringing in some local code would be a sufficient, but only temporary, workaround for the time being.

10

Thank you Steven. No, I am running the singularity image from a HPC account where freesurfer is not installed and I can not install it easily since I have ho administrator rights on that system.

Unfortunately I am still stuck with that problem. One other idea was that it may be coming from the fact that my subjects have sessions. qsiprep output does have the session mentioned, but not the freesurfer output, this may be confusing the reconstruction process?

11

What is the structure of your fsdir? The usual *-prep pipelines make one FreeSurfer output even if there are multiple sessions. QSIPrep is expecting all the FS outputs to be in $FSDIR/$SUB , with no additional sub-folders (e.g. $FSDIR/$SUB/$SES or $FSDIR/$SUB-$SES is not allowed).

12

Thank you again Steven. I think I found what I was doing wrong. I got confused when I read that fs-recon was excised out of QSIPREP and needed to be done with an external software. I thought that implied to use the --dwi-only option.

When using the QSIPREP preprocessing output that was run without the --dwi-only option, the recon is now going further.

Thanks for the help!

1 Like
13

Explanation of the accidentally discovered bug here: Help with qsiprep reconstruction pipeline command · Issue #432 · PennLINC/qsiprep · GitHub

1 Like
15

Hi @XinWen08297173,

This does not appear to be the same issue, as the original post had to do with the reconstruction part of qsiprep, which happens after preprocessing. The original post also was specific with the dwi-only flag, which was not the case for you. Closing this issue as it has been solved, and we can continue the discussion on the post you already made.

Best,
Steven

16