Error when running xcp-d 1.1.3 ("list index out of range")

Hello,
I have been running xcp_d on a computing cluster without any problems. However, I wanted to change my pipeline to incorporate filtering of motion regressors to account for respiratory artifact. I ran this in the version that I was using (0.0.8), and it generated errors. I thought I would rerun in the latest version (1.1.3), to see if this would fix the errors. With the new version, I am getting an error very early on about the h5 file (“list index out of range”). Could this be related to compatibility with the fmriprep version? I am using a relatively up-to-date fmriprep (1.2.14). (EDITED to correct: fmriprep version 20.2.6). Thanks in advance for your help!

Best,
Danella

Here is the code that I ran:

singularity run -B /ix/dhafeman /ix/dhafeman/singularity/xcp_d-0.1.3.simg /ix/dhafeman/BDLONG/Preproc/fmriprep /ix/dhafeman/BDLONG/Preproc participant -f 0.1 -w /ix/dhafeman/BDLONG/Work --smoothing 8 --despike -p 36P --lower-bpf .009 --upper-bpf .08 --motion-filter-type notch --band-stop-min 12 --band-stop-max 20 --participant_label 400

And the error:

Matplotlib created a temporary config/cache directory at /scratch/slurm-6949518/matplotlib-wo8bz5nf because the default path (/home/xcp_d/.config/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
220929-11:50:00,755 nipype.workflow IMPORTANT:

Running xcp_d version 0.1.3+0.g9fec4d8.dirty:
  * fMRI directory path: /ix/dhafeman/BDLONG/Preproc/fmriprep.
  * Participant list: ['400'].
  * Run identifier: 20220929-114757_bb881ab7-7621-4de3-8977-bb519b7db393.

220929-11:51:51,557 nipype.utils WARNING:
A newer version (1.8.4) of nipy/nipype is available. You are using 1.8.3
Process Process-2:
Traceback (most recent call last):
File “/usr/local/miniconda/lib/python3.8/multiprocessing/process.py”, line 315, in _bootstrap
self.run()
File “/usr/local/miniconda/lib/python3.8/multiprocessing/process.py”, line 108, in run
self._target(*self._args, **self._kwargs)
File “/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/cli/run.py”, line 617, in build_workflow
retval[‘workflow’] = init_xcpd_wf(
File “/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/workflow/base.py”, line 148, in init_xcpd_wf
single_subj_wf = init_subject_wf(
File “/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/workflow/base.py”, line 329, in init_subject_wf
anatomical_wf = init_anatomical_wf(
File “/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/workflow/anatomical.py”, line 364, in init_anatomical_wf
h5_file = fnmatch.filter(
IndexError: list index out of range

Hello Danella,

I don’t see fMRIPrep 1.2.14 in the list of versions in the changelog, but fmriprep 1.2.X is ~3 years old, and recent versions of fMRIPrep has gone through many changes, including in output file formatting. This could include anatomical transformations stored as .h5 files that XCP seems to be expecting based on the error. I would suggest running a subject in the latest fmriprep release and testing xcp_d on that subject.

Best,
Steven

Apologies - I misspoke. The version is actually 20.2.6. (I was running via flywheel, and that was the version 1.2.14.) So I believe this is a fairly up-do-date version? Thanks!!

Hi Steven, thanks for the response, I think I see what’s going on now. Relevant piece of source code is searching for wildcard containing MNI152NLin6Asym

h5_file = fnmatch.filter( all_files, “sub-” + subject_id + “*from-T1w_to-MNI152NLin6Asym_mode-image_xfm.h5”,)[0] # MB

But our .h5 files have filenames that go something like this sub-400_ses-01_acq-mprage_from-T1w_to_MNI152NLin2009cAsym_mode-image_xfm.h5

Hi,

It would help to know what your fMRIPrep command was to get an expectation of what XCP_D should be doing. The code you referenced, from this function, seems to be part of a CIFTI-based workflow. Did you create CIFTIs in your fmriprep command (which includes having MNI152NLin6Asym as an output space)?

Best,
Steven

The analysis was run in flywheel. I rechecked the configuration, and output-spaces are both MNI152NLin2009cAsym and T1w. I did not run the cifti option. Is it possible that the error is related to T1w space? I will run another subject that I did not analyze in T1w space, and see if that solves the issue. It sounds like this also might be something that I should ask the flywheel support about.

Was freesurfer run as part of fmriprep? Knowing the full fmriprep options would help.

Yes, it was. Here is the configuration from the gear log. Please let me know if any other info would be helpful - I really appreciate your help with this!
Best,
Danella

Gear Name: bids-fmriprep, Gear Version: 1.2.14_20.2.6
Executor: analysis-fca40f23, CPU: 4 cores, Memory: 27GB, Disk: 208GB, Swap: 32GB
Gear starting…
INFO:main:Running gear in /flywheel/v0
INFO:flywheel_gear_toolkit.context.context:Destination is analysis=62c871bbd13b95142069f2bb
INFO:flywheel_gear_toolkit.context.context:Input file “freesurfer_license” is freesurfer_license.txt from project=5d37d64952f58b0031c65ac3
INFO:flywheel_gear_toolkit.context.context:Config “anat-only=False”
INFO:flywheel_gear_toolkit.context.context:Config “aroma-melodic-dimensionality=-200”
INFO:flywheel_gear_toolkit.context.context:Config “boilerplate_only=False”
INFO:flywheel_gear_toolkit.context.context:Config “bold2t1w-dof=6”
INFO:flywheel_gear_toolkit.context.context:Config “bold2t1w-init=register”
INFO:flywheel_gear_toolkit.context.context:Config “cifti-output=False”
INFO:flywheel_gear_toolkit.context.context:Config “debug=”
INFO:flywheel_gear_toolkit.context.context:Config “dvars-spike-threshold=1.5”
INFO:flywheel_gear_toolkit.context.context:Config “echo-idx=”
INFO:flywheel_gear_toolkit.context.context:Config “error-on-aroma-warnings=False”
INFO:flywheel_gear_toolkit.context.context:Config “fd-spike-threshold=0.5”
INFO:flywheel_gear_toolkit.context.context:Config “fmap-bspline=False”
INFO:flywheel_gear_toolkit.context.context:Config “fmap-no-demean=False”
INFO:flywheel_gear_toolkit.context.context:Config “force-bbr=False”
INFO:flywheel_gear_toolkit.context.context:Config “force-no-bbr=False”
INFO:flywheel_gear_toolkit.context.context:Config “force-syn=False”
INFO:flywheel_gear_toolkit.context.context:Config “fs-no-reconall=False”
INFO:flywheel_gear_toolkit.context.context:Config “gear-FREESURFER_LICENSE=”
INFO:flywheel_gear_toolkit.context.context:Config “gear-dry-run=False”
INFO:flywheel_gear_toolkit.context.context:Config “gear-intermediate-files=”
INFO:flywheel_gear_toolkit.context.context:Config “gear-intermediate-folders=”
INFO:flywheel_gear_toolkit.context.context:Config “gear-keep-fsaverage=False”
INFO:flywheel_gear_toolkit.context.context:Config “gear-keep-output=False”
INFO:flywheel_gear_toolkit.context.context:Config “gear-log-level=INFO”
INFO:flywheel_gear_toolkit.context.context:Config “gear-log-to-file=False”
INFO:flywheel_gear_toolkit.context.context:Config “gear-run-bids-validation=False”
INFO:flywheel_gear_toolkit.context.context:Config “gear-save-intermediate-output=False”
INFO:flywheel_gear_toolkit.context.context:Config “gear-writable-dir=/var/tmp”
INFO:flywheel_gear_toolkit.context.context:Config “ignore=”
INFO:flywheel_gear_toolkit.context.context:Config “longitudinal=False”
INFO:flywheel_gear_toolkit.context.context:Config “lsf-cpu=4”
INFO:flywheel_gear_toolkit.context.context:Config “lsf-ram=rusage[mem=12000]”
INFO:flywheel_gear_toolkit.context.context:Config “md-only-boilerplate=False”
INFO:flywheel_gear_toolkit.context.context:Config “medial-surface-nan=False”
INFO:flywheel_gear_toolkit.context.context:Config “mem_mb=0”
INFO:flywheel_gear_toolkit.context.context:Config “n_cpus=0”
INFO:flywheel_gear_toolkit.context.context:Config “no-submm-recon=False”
INFO:flywheel_gear_toolkit.context.context:Config “notrack=False”
INFO:flywheel_gear_toolkit.context.context:Config “omp-nthreads=0”
INFO:flywheel_gear_toolkit.context.context:Config “output-spaces=MNI152NLin2009cAsym T1w”
INFO:flywheel_gear_toolkit.context.context:Config “reports-only=False”
INFO:flywheel_gear_toolkit.context.context:Config “resource-monitor=False”
INFO:flywheel_gear_toolkit.context.context:Config “return-all-components=False”
INFO:flywheel_gear_toolkit.context.context:Config “singularity-debug=False”
INFO:flywheel_gear_toolkit.context.context:Config “skip-bids-validation=False”
INFO:flywheel_gear_toolkit.context.context:Config “skull-strip-fixed-seed=False”
INFO:flywheel_gear_toolkit.context.context:Config “skull-strip-t1w=force”
INFO:flywheel_gear_toolkit.context.context:Config “skull-strip-template=OASIS30ANTs”
INFO:flywheel_gear_toolkit.context.context:Config “stop-on-first-crash=False”
INFO:flywheel_gear_toolkit.context.context:Config “task-id=”
INFO:flywheel_gear_toolkit.context.context:Config “use-aroma=False”
INFO:flywheel_gear_toolkit.context.context:Config “use-syn-sdc=False”
INFO:flywheel_gear_toolkit.context.context:Config “verbose=”
INFO:flywheel_gear_toolkit.context.context:Config “write-graph=False”

Hi @Danella1 and @TylerB,

See here: Error in finding template space of the input · Issue #556 · PennLINC/xcp_d · GitHub (thanks to @jsein for opening the issue; related to Issue with XCP_D on HPC and TemplateFlow - #13 by jsein). In short, you need certain output spaces from fmriprep (and as of now, NOT other spaces; will be fixed) to run XCP_D.

Best,
Steven

Thanks for looking into this, and this seemed very likely. However, I have rerun fmriprep on flywheel without specifying the T1w space. In this case, the space is:

output_spaces = “MNI152NLin2009cAsym:res-native”

I checked that there are no other spaces (i.e. T1w) in the preprocessed directory. Do you have any additional ideas? I am not sure if it would be better to specify the resolution, instead of leave it in native space - could that be causing the problem? I can also ask the flywheel help desk if they have any ideas.

Thanks!
Danella

The GitHub issue said XCP_D requires MNI152NLin6Asym

1 Like

Ahh, I see. I had focused on the absence of T1w - but not the necessary addition of MNI152NLin6Asym. I will rerun with this space included. Thanks!