Fmriprep v.21.0.1 wf.output_select error

Summary of what happened:

Hello community,
I am running fmriprep v.21.0.1 on a few subjects that each have 4 task runs. I am getting an error as described below.

Command used (and if a helper script was used, a link to the helper script or the command generated):

This is my command:

singularity run -B /PATH/TO/DATABASE/:/work -e /PATH/TO/CONTAINERS/nipreps_fmriprep_21.0.1.sif \
/work/3T/bids /work/3T/derivatives/fmriprep-21.0.1/ \
participant --participant-label 3T012 3T013 3T014 \
-w /work/3T/derivatives/fmriprep-temp \
--output-spaces T1w --skull-strip-t1w skip \
--fd-spike-threshold 0.5 --dvars-spike-threshold 2.0 --bold2t1w-dof 9 \
--return-all-components --mem-mb 60000 --omp-nthreads 14 --nthreads 16 \
--fs-no-reconall --fs-subjects-dir /work/3T/derivatives/freesurfer --fs-license-file /work/3T/code/license.txt

Version:

21.0.1

Environment (Docker, Singularity, custom installation):

Singularity

Data formatted according to a validatable standard? Please provide the output of the validator:

Relevant log outputs (up to 20 lines):

I am encountering the following wf.output_select error on two of four task runs of just one subject.

Node Name: fmriprep_wf.single_subject_3T013_wf.func_preproc_task_InterceptionPhase3_wf.output_select

File: /work/3T/derivatives/fmriprep-21.0.1/sub-3T013/log/20230526-103626_3c2633f5-d116-4d86-8194-f373707bd39f/crash-20230526-111528-kpunjabi-output_select-a4d173f4-f91d-4497-b713-807ffad66c31.txt
Working Directory: /work/3T/derivatives/fmriprep-temp/fmriprep_wf/single_subject_3T013_wf/func_preproc_task_InterceptionPhase3_wf/output_select
Inputs:
fmap: ['/work/3T/derivatives/fmriprep-temp/fmriprep_wf/single_subject_3T013_wf/fmap_preproc_wf/wf_auto_00000/topup/sub-3T013_dir-AP_epi_idx-000_merged_field.nii.gz']
fmap_coeff: ['/work/3T/derivatives/fmriprep-temp/fmriprep_wf/single_subject_3T013_wf/fmap_preproc_wf/wf_auto_00000/fix_coeff/sub-3T013_dir-AP_epi_idx-000_merged_base_fieldcoef_fixed.nii.gz']
fmap_mask: ['/work/3T/derivatives/fmriprep-temp/fmriprep_wf/single_subject_3T013_wf/fmap_preproc_wf/wf_auto_00000/brainextraction_wf/masker/clipped_mask.nii.gz']
fmap_ref: ['/work/3T/derivatives/fmriprep-temp/fmriprep_wf/single_subject_3T013_wf/fmap_preproc_wf/wf_auto_00000/brainextraction_wf/clipper_post/clipped.nii.gz']
key: B0map
keys: ['auto_00000']
sdc_method: ['PEB/PEPOLAR (phase-encoding based / PE-POLARity)']
Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/site-packages/nipype/pipeline/plugins/multiproc.py", line 344, in _send_procs_to_workers
    self.procs[jobid].run(updatehash=updatehash)
  File "/opt/conda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 516, in run
    result = self._run_interface(execute=True)
  File "/opt/conda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 635, in _run_interface
    return self._run_command(execute)
  File "/opt/conda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 741, in _run_command
    result = self._interface.run(cwd=outdir)
  File "/opt/conda/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 430, in run
    outputs = self.aggregate_outputs(runtime)
  File "/opt/conda/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 506, in aggregate_outputs
    predicted_outputs = self._list_outputs()  # Predictions from _list_outputs
  File "/opt/conda/lib/python3.8/site-packages/niworkflows/interfaces/utility.py", line 217, in _list_outputs
    index = self.inputs.keys.index(self.inputs.key)
ValueError: 'B0map' is not in list

Screenshots / relevant information:

Any help in troubleshooting this is appreciated.
Thanks,
Khushboo

Hi @kpunjabi,

I have relabeled your post as Software Support and restructured your post according to the software support category template. Please post software related questions under this category in the future.

Some additional points:

  1. Are your data BIDS valid? (feel free to edit your original post to include this information).
  2. Does this error persist on the most recent version of fMRIPrep (23.0.2 at this time)?
  3. The fs-no-reconall flag is not recommended (see here)
  4. This looks like part of the fieldmap processing scheme. How are you associating fieldmaps with BOLD files?

Best,
Steven

Hi Steven,
Thanks for moving and reorganizing my post.

To answer your questions:

  1. Yes, the dataset is online bids-validated.
  2. I am currently starting a run on fmriprep 23.0.2 to check if the error persists.
  3. I run freesurfer outside the fmriprep container because I use FS v.7.1.1 which seems to give me fewer errors in wm.mgz as compared to v6, the fmriprep container has 6.0.1. I manually correct these errors to get better surfaces.
  4. I have AP PA fieldmaps that have intended for fields in the jsons:
"IntendedFor": [
    "func/sub-3T013_task-InterceptionPhase1_bold.nii.gz",
    "func/sub-3T013_task-InterceptionPhase2_bold.nii.gz",
    "func/sub-3T013_task-InterceptionPhase3_bold.nii.gz",
    "func/sub-3T013_task-InterceptionPhase4_bold.nii.gz"
  ]

The error occurs for sub-3T013_task-InterceptionPhase3_bold.nii.gz and sub-3T013_task-InterceptionPhase4_bold.nii.gz. This is the tree for this subject’s bids folder:

├── anat
│   ├── sub-3T013_acq-csmp2rage_inv-1_part-mag_MP2RAGE.json
│   ├── sub-3T013_acq-csmp2rage_inv-1_part-mag_MP2RAGE.nii.gz
│   ├── sub-3T013_acq-csmp2rage_inv-1_part-phase_MP2RAGE.json
│   ├── sub-3T013_acq-csmp2rage_inv-1_part-phase_MP2RAGE.nii.gz
│   ├── sub-3T013_acq-csmp2rage_inv-2_part-mag_MP2RAGE.json
│   ├── sub-3T013_acq-csmp2rage_inv-2_part-mag_MP2RAGE.nii.gz
│   ├── sub-3T013_acq-csmp2rage_inv-2_part-phase_MP2RAGE.json
│   ├── sub-3T013_acq-csmp2rage_inv-2_part-phase_MP2RAGE.nii.gz
│   ├── sub-3T013_acq-uni_rec-defdenSS_T1w.json
│   └── sub-3T013_acq-uni_rec-defdenSS_T1w.nii
├── dwi
│   ├── sub-3T013_dir-AP_dwi.bval
│   ├── sub-3T013_dir-AP_dwi.bvec
│   ├── sub-3T013_dir-AP_dwi.json
│   ├── sub-3T013_dir-AP_dwi.nii.gz
│   ├── sub-3T013_dir-PA_dwi.bval
│   ├── sub-3T013_dir-PA_dwi.bvec
│   ├── sub-3T013_dir-PA_dwi.json
│   └── sub-3T013_dir-PA_dwi.nii.gz
├── fmap
│   ├── sub-3T013_dir-AP_epi.json
│   ├── sub-3T013_dir-AP_epi.nii.gz
│   ├── sub-3T013_dir-PA_epi.json
│   └── sub-3T013_dir-PA_epi.nii.gz
└── func
    ├── sub-3T013_task-InterceptionPhase1_bold.json
    ├── sub-3T013_task-InterceptionPhase1_bold.nii.gz
    ├── sub-3T013_task-InterceptionPhase1_recording-cardiac_physio.json
    ├── sub-3T013_task-InterceptionPhase1_recording-cardiac_physio.tsv.gz
    ├── sub-3T013_task-InterceptionPhase1_recording-respiratory_physio.json
    ├── sub-3T013_task-InterceptionPhase1_recording-respiratory_physio.tsv.gz
    ├── sub-3T013_task-InterceptionPhase1_sbref.json
    ├── sub-3T013_task-InterceptionPhase1_sbref.nii.gz
    ├── sub-3T013_task-InterceptionPhase2_bold.json
    ├── sub-3T013_task-InterceptionPhase2_bold.nii.gz
    ├── sub-3T013_task-InterceptionPhase2_recording-cardiac_physio.json
    ├── sub-3T013_task-InterceptionPhase2_recording-cardiac_physio.tsv.gz
    ├── sub-3T013_task-InterceptionPhase2_recording-respiratory_physio.json
    ├── sub-3T013_task-InterceptionPhase2_recording-respiratory_physio.tsv.gz
    ├── sub-3T013_task-InterceptionPhase2_sbref.json
    ├── sub-3T013_task-InterceptionPhase2_sbref.nii.gz
    ├── sub-3T013_task-InterceptionPhase3_bold.json
    ├── sub-3T013_task-InterceptionPhase3_bold.nii.gz
    ├── sub-3T013_task-InterceptionPhase3_recording-cardiac_physio.json
    ├── sub-3T013_task-InterceptionPhase3_recording-cardiac_physio.tsv.gz
    ├── sub-3T013_task-InterceptionPhase3_recording-respiratory_physio.json
    ├── sub-3T013_task-InterceptionPhase3_recording-respiratory_physio.tsv.gz
    ├── sub-3T013_task-InterceptionPhase3_sbref.json
    ├── sub-3T013_task-InterceptionPhase3_sbref.nii.gz
    ├── sub-3T013_task-InterceptionPhase4_bold.json
    ├── sub-3T013_task-InterceptionPhase4_bold.nii.gz
    ├── sub-3T013_task-InterceptionPhase4_recording-cardiac_physio.json
    ├── sub-3T013_task-InterceptionPhase4_recording-cardiac_physio.tsv.gz
    ├── sub-3T013_task-InterceptionPhase4_recording-respiratory_physio.json
    ├── sub-3T013_task-InterceptionPhase4_recording-respiratory_physio.tsv.gz
    ├── sub-3T013_task-InterceptionPhase4_sbref.json
    └── sub-3T013_task-InterceptionPhase4_sbref.nii.gz

This is the tree of the fmriprep output after the crash

├── anat
│   ├── sub-3T013_acq-uni_rec-defdenSS_desc-brain_mask.json
│   ├── sub-3T013_acq-uni_rec-defdenSS_desc-brain_mask.nii.gz
│   ├── sub-3T013_acq-uni_rec-defdenSS_desc-preproc_T1w.json
│   ├── sub-3T013_acq-uni_rec-defdenSS_desc-preproc_T1w.nii.gz
│   ├── sub-3T013_acq-uni_rec-defdenSS_dseg.nii.gz
│   ├── sub-3T013_acq-uni_rec-defdenSS_from-MNI152NLin2009cAsym_to-T1w_mode-image_xfm.h5
│   ├── sub-3T013_acq-uni_rec-defdenSS_from-T1w_to-MNI152NLin2009cAsym_mode-image_xfm.h5
│   ├── sub-3T013_acq-uni_rec-defdenSS_label-CSF_probseg.nii.gz
│   ├── sub-3T013_acq-uni_rec-defdenSS_label-GM_probseg.nii.gz
│   └── sub-3T013_acq-uni_rec-defdenSS_label-WM_probseg.nii.gz
├── figures
│   ├── sub-3T013_acq-uni_rec-defdenSS_desc-conform_T1w.html
│   ├── sub-3T013_acq-uni_rec-defdenSS_dseg.svg
│   ├── sub-3T013_acq-uni_rec-defdenSS_space-MNI152NLin2009cAsym_T1w.svg
│   ├── sub-3T013_desc-about_T1w.html
│   ├── sub-3T013_desc-summary_T1w.html
│   ├── sub-3T013_fmapid-auto00000_desc-pepolar_fieldmap.svg
│   ├── sub-3T013_task-InterceptionPhase1_desc-carpetplot_bold.svg
│   ├── sub-3T013_task-InterceptionPhase1_desc-compcorvar_bold.svg
│   ├── sub-3T013_task-InterceptionPhase1_desc-confoundcorr_bold.svg
│   ├── sub-3T013_task-InterceptionPhase1_desc-flirtbbr_bold.svg
│   ├── sub-3T013_task-InterceptionPhase1_desc-rois_bold.svg
│   ├── sub-3T013_task-InterceptionPhase1_desc-sdc_bold.svg
│   ├── sub-3T013_task-InterceptionPhase1_desc-summary_bold.html
│   ├── sub-3T013_task-InterceptionPhase1_desc-validation_bold.html
│   ├── sub-3T013_task-InterceptionPhase2_desc-carpetplot_bold.svg
│   ├── sub-3T013_task-InterceptionPhase2_desc-compcorvar_bold.svg
│   ├── sub-3T013_task-InterceptionPhase2_desc-confoundcorr_bold.svg
│   ├── sub-3T013_task-InterceptionPhase2_desc-flirtbbr_bold.svg
│   ├── sub-3T013_task-InterceptionPhase2_desc-rois_bold.svg
│   ├── sub-3T013_task-InterceptionPhase2_desc-sdc_bold.svg
│   ├── sub-3T013_task-InterceptionPhase2_desc-summary_bold.html
│   ├── sub-3T013_task-InterceptionPhase2_desc-validation_bold.html
│   ├── sub-3T013_task-InterceptionPhase3_desc-validation_bold.html
│   └── sub-3T013_task-InterceptionPhase4_desc-validation_bold.html
├── fmap
│   ├── sub-3T013_fmapid-auto00000_desc-coeff_fieldmap.nii.gz
│   ├── sub-3T013_fmapid-auto00000_desc-epi_fieldmap.nii.gz
│   ├── sub-3T013_fmapid-auto00000_desc-preproc_fieldmap.json
│   └── sub-3T013_fmapid-auto00000_desc-preproc_fieldmap.nii.gz
├── func
│   ├── sub-3T013_task-InterceptionPhase1_desc-confounds_timeseries.json
│   ├── sub-3T013_task-InterceptionPhase1_desc-confounds_timeseries.tsv
│   ├── sub-3T013_task-InterceptionPhase1_from-scanner_to-T1w_mode-image_xfm.txt
│   ├── sub-3T013_task-InterceptionPhase1_from-T1w_to-scanner_mode-image_xfm.txt
│   ├── sub-3T013_task-InterceptionPhase1_space-T1w_boldref.nii.gz
│   ├── sub-3T013_task-InterceptionPhase1_space-T1w_desc-brain_mask.json
│   ├── sub-3T013_task-InterceptionPhase1_space-T1w_desc-brain_mask.nii.gz
│   ├── sub-3T013_task-InterceptionPhase1_space-T1w_desc-preproc_bold.json
│   ├── sub-3T013_task-InterceptionPhase1_space-T1w_desc-preproc_bold.nii.gz
│   ├── sub-3T013_task-InterceptionPhase2_desc-confounds_timeseries.json
│   ├── sub-3T013_task-InterceptionPhase2_desc-confounds_timeseries.tsv
│   ├── sub-3T013_task-InterceptionPhase2_from-scanner_to-T1w_mode-image_xfm.txt
│   ├── sub-3T013_task-InterceptionPhase2_from-T1w_to-scanner_mode-image_xfm.txt
│   ├── sub-3T013_task-InterceptionPhase2_space-T1w_boldref.nii.gz
│   ├── sub-3T013_task-InterceptionPhase2_space-T1w_desc-brain_mask.json
│   ├── sub-3T013_task-InterceptionPhase2_space-T1w_desc-brain_mask.nii.gz
│   ├── sub-3T013_task-InterceptionPhase2_space-T1w_desc-preproc_bold.json
│   └── sub-3T013_task-InterceptionPhase2_space-T1w_desc-preproc_bold.nii.gz
└── log
    └── 20230526-103626_3c2633f5-d116-4d86-8194-f373707bd39f
        ├── crash-20230526-111528-kpunjabi-output_select-a4d173f4-f91d-4497-b713-807ffad66c31.txt
        ├── crash-20230526-111528-kpunjabi-output_select-df10755d-736d-4824-82f2-48c0c727953f.txt
        └── fmriprep.toml

The same workflow error persists with 23.0.2, with the same 2 images:

Node Name: fmriprep_23_0_wf.single_subject_3T013_wf.func_preproc_task_InterceptionPhase3_wf.output_select

File: /work/3T/derivatives/fmriprep-23.0.2/sub-3T013/log/20230530-155040_f535e9e2-47b4-46a5-a17a-878572ae9fc6/crash-20230530-162838-kpunjabi-output_select-c367e4e5-0816-4089-a0b4-ae4ce53e3871.txt
Working Directory: /work/3T/derivatives/fmriprep-temp23/fmriprep_23_0_wf/single_subject_3T013_wf/func_preproc_task_InterceptionPhase3_wf/output_select
Inputs:
fmap: ['/work/3T/derivatives/fmriprep-temp23/fmriprep_23_0_wf/single_subject_3T013_wf/fmap_preproc_wf/wf_auto_00000/from_las_fmap/reoriented.nii.gz']
fmap_coeff: ['/work/3T/derivatives/fmriprep-temp23/fmriprep_23_0_wf/single_subject_3T013_wf/fmap_preproc_wf/wf_auto_00000/fix_coeff/sub-3T013_dir-PA_epi_average_merged_sliced_volreg_base_fieldcoef_fixed.nii.gz']
fmap_mask: ['/work/3T/derivatives/fmriprep-temp23/fmriprep_23_0_wf/single_subject_3T013_wf/fmap_preproc_wf/wf_auto_00000/brainextraction_wf/masker/clipped_mask.nii.gz']
fmap_ref: ['/work/3T/derivatives/fmriprep-temp23/fmriprep_23_0_wf/single_subject_3T013_wf/fmap_preproc_wf/wf_auto_00000/brainextraction_wf/clipper_post/clipped.nii.gz']
key: B0map
keys: ['auto_00000']
sdc_method: ['PEB/PEPOLAR (phase-encoding based / PE-POLARity)']
Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/plugins/multiproc.py", line 344, in _send_procs_to_workers
    self.procs[jobid].run(updatehash=updatehash)
  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node output_select.

Traceback:
	Traceback (most recent call last):
	  File "/opt/conda/lib/python3.9/site-packages/nipype/interfaces/base/core.py", line 400, in run
	    outputs = self.aggregate_outputs(runtime)
	  File "/opt/conda/lib/python3.9/site-packages/nipype/interfaces/base/core.py", line 429, in aggregate_outputs
	    predicted_outputs = self._list_outputs()  # Predictions from _list_outputs
	  File "/opt/conda/lib/python3.9/site-packages/niworkflows/interfaces/utility.py", line 217, in _list_outputs
	    index = self.inputs.keys.index(self.inputs.key)
	ValueError: 'B0map' is not in list

It turns out that there was a mistake in the JSON files of each bold image of the problematic runs:
For each of them, there was: "B0FieldSource": "B0map" at the end of the JSON file, with no corresponding B0FieldIdentifier anywhere for this subject. It was a mistake when preparing the BIDS dataset, we moved from using B0FieldSource/B0FieldIdentifier to using IntendedFor only but forgot to remove some occurrences of "B0FieldSource".

Interestingly, as @kpunjabi mentioned, this mistake did not trigger an error nor a warning from the BIDS validator (I checked with bids-validator@1.9.3).