Issue with XCP_D on HPC and TemplateFlow


I am using: xcp_d version 0.1.3+0.g9fec4d8.dirty with singularity on a HPC.

Here is the command I used:

singularity run -B /scratch/jsein/BIDS:/work,$HOME/.templateflow:$HOME/.cache/templateflow  \
   /scratch/jsein/my_images/xcp_d-0.1.3.simg \
   /work/$study/derivatives/fmriprep /work/$study/derivatives/xcpd \
   participant --participant-label $sub \
   -w /work/temp_data_${study}\
   --mem_gb 50 --omp-nthreads 10 --nthreads 12 --smoothing 4  \
   --nuissance-regressors acompcor --despike --lower-bpf 0.01 --upper-bpf 0.08 

I am interested in testing XCP_D on our data but I am stuck right at the beginning with this error which I believe is linked to the fact that the singularity container does not have access to the internet on the HPC I am using it:


  0%|          | 0.00/28.0 [00:00<?, ?B/s]
29.0B [00:00, 324B/s]                     
Process Process-2:
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.8/multiprocessing/", line 315, in _bootstrap
  File "/usr/local/miniconda/lib/python3.8/multiprocessing/", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/cli/", line 617, in build_workflow
    retval['workflow'] = init_xcpd_wf(
  File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/workflow/", line 148, in init_xcpd_wf
    single_subj_wf = init_subject_wf(
  File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/workflow/", line 395, in init_subject_wf
    bold_postproc_wf = init_boldpostprocess_wf(
  File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/workflow/", line 341, in init_boldpostprocess_wf
    bold2MNI_trans, bold2T1w_trans = get_transformfilex(
  File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/utils/", line 87, in get_transformfilex
    t1w_to_mni = glob.glob(mnisf + 'from-T1w_to-' + template +
UnboundLocalError: local variable 'template' referenced before assignment

I know how to deal with it with FMRIPREP:

But for the moment I don’t know how to deal with this issue with XCP_D. Can anyone help me?

Thank you!


Hi Julien,

What output spaces did you use in fMRIPrep? Would you mind sharing your full command?


Hi Steven,

Thank you for passing by!

Sure, here is the fmriprep command I used:

singularity run -B /scratch/jsein/BIDS:/work,$HOME/.templateflow:/opt/templateflow --cleanenv /scratch/jsein/my_images/fmriprep-21.0.2.simg \
		 --fs-license-file /work/freesurfer/license.txt /work/$study /work/$study/derivatives/fmriprep_nosbref  \
		 participant --participant-label $sub \
		 -w /work/temp_data_${study}\
		 --mem-mb 50000 --omp-nthreads 10 --nthreads 12  \
		 --fd-spike-threshold 0.5 --dvars-spike-threshold 2.0 --bold2t1w-dof 9 --cifti-output 91k   \
		 --output-spaces MNI152NLin2009cAsym MNI152NLin6Asym T1w --ignore slicetiming  --fs-subjects-dir /work/$study/derivatives/fmriprep/sourcedata/freesurfer

Can you print the contents of a subject’s anatomical fmriprep outputs?

Here it is:

β”œβ”€β”€ sub-01_desc-aparcaseg_dseg.nii.gz
β”œβ”€β”€ sub-01_desc-aseg_dseg.nii.gz
β”œβ”€β”€ sub-01_desc-brain_mask.json
β”œβ”€β”€ sub-01_desc-brain_mask.nii.gz
β”œβ”€β”€ sub-01_desc-preproc_T1w.json
β”œβ”€β”€ sub-01_desc-preproc_T1w.nii.gz
β”œβ”€β”€ sub-01_dseg.nii.gz
β”œβ”€β”€ sub-01_from-MNI152NLin2009cAsym_to-T1w_mode-image_xfm.h5
β”œβ”€β”€ sub-01_from-MNI152NLin6Asym_to-T1w_mode-image_xfm.h5
β”œβ”€β”€ sub-01_from-T1w_to-MNI152NLin2009cAsym_mode-image_xfm.h5
β”œβ”€β”€ sub-01_from-T1w_to-MNI152NLin6Asym_mode-image_xfm.h5
β”œβ”€β”€ sub-01_from-T1w_to-fsnative_mode-image_xfm.txt
β”œβ”€β”€ sub-01_from-fsnative_to-T1w_mode-image_xfm.txt
β”œβ”€β”€ sub-01_label-CSF_probseg.nii.gz
β”œβ”€β”€ sub-01_label-GM_probseg.nii.gz
β”œβ”€β”€ sub-01_label-WM_probseg.nii.gz
β”œβ”€β”€ sub-01_space-MNI152NLin2009cAsym_desc-brain_mask.json
β”œβ”€β”€ sub-01_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz
β”œβ”€β”€ sub-01_space-MNI152NLin2009cAsym_desc-preproc_T1w.json
β”œβ”€β”€ sub-01_space-MNI152NLin2009cAsym_desc-preproc_T1w.nii.gz
β”œβ”€β”€ sub-01_space-MNI152NLin2009cAsym_dseg.nii.gz
β”œβ”€β”€ sub-01_space-MNI152NLin2009cAsym_label-CSF_probseg.nii.gz
β”œβ”€β”€ sub-01_space-MNI152NLin2009cAsym_label-GM_probseg.nii.gz
β”œβ”€β”€ sub-01_space-MNI152NLin2009cAsym_label-WM_probseg.nii.gz
β”œβ”€β”€ sub-01_space-MNI152NLin6Asym_desc-brain_mask.json
β”œβ”€β”€ sub-01_space-MNI152NLin6Asym_desc-brain_mask.nii.gz
β”œβ”€β”€ sub-01_space-MNI152NLin6Asym_desc-preproc_T1w.json
β”œβ”€β”€ sub-01_space-MNI152NLin6Asym_desc-preproc_T1w.nii.gz
β”œβ”€β”€ sub-01_space-MNI152NLin6Asym_dseg.nii.gz
β”œβ”€β”€ sub-01_space-MNI152NLin6Asym_label-CSF_probseg.nii.gz
β”œβ”€β”€ sub-01_space-MNI152NLin6Asym_label-GM_probseg.nii.gz
└── sub-01_space-MNI152NLin6Asym_label-WM_probseg.nii.gz

I do not think this issue is related to templateflow, but rather to the naming convention of the .h5 transformation files. That is, looking at the code here (xcp_d/ at 8aa79b9ba4612fbf38231886ee9ac71215ec88e3 Β· PennLINC/xcp_d Β· GitHub), there are different conditions that they use to try to figure out what template is being used based on the transformation name, but somehow, it is not satisfying any of them, leading to the undefined variable error.

Hmm, the h5 files look fine, I guess it would help to know what h5 file is being checked, which would mean introducing print statements in the utils command linked above. You could try building the singularity container as a sandbox and making the changes yourself, but I wish the error would be more obvious :cold_sweat:

Thank you for this feedback.

I had an issue for another study where only MNI152NLin2009cAsym was asked as an output for fmriprep so I tried for another when where the MNI152NLin2009cAsym MNI152NLin6Asym T1w were called for the output.

I am puzzled still by this output in the error log:


  0%|          | 0.00/28.0 [00:00<?, ?B/s]
29.0B [00:00, 324B/s]      

which seems to indicate that XCP_D tried to download a template from TemplateFlow.

I think this case warrants a github issues post, as it is may be due to the code not filtering the transform files properly as opposed to any user error. Hope you can get to the bottom of this!

Thank you again!
ok I will dig deeper into this, I will let you know in this thread if I find a solution.

I agree, I will post an issue on GitHub. As it was my first trial with the XCP_D toolbox for which I read good recommendations, I was wondering if I was missing something obvious in my command line.

For information, here is the output for a subject for which only MNI152NLin2009cAsym was asked for as output-spaces in fmriprep (v21.0.2):

Process Process-2:
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.8/multiprocessing/", line 315, in _bootstrap
  File "/usr/local/miniconda/lib/python3.8/multiprocessing/", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/cli/", line 617, in build_workflow
    retval['workflow'] = init_xcpd_wf(
  File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/workflow/", line 148, in init_xcpd_wf
    single_subj_wf = init_subject_wf(
  File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/workflow/", line 329, in init_subject_wf
    anatomical_wf = init_anatomical_wf(
  File "/usr/local/miniconda/lib/python3.8/site-packages/xcp_d/workflow/", line 364, in init_anatomical_wf
    h5_file = fnmatch.filter(
IndexError: list index out of range

At the very least this seems to add evidence that this is a H5 finding problem as opposed to a templateflow issue.

I created a Github issue on this topic:

1 Like