FileNotFoundError: No such file or directory [...] for output 'output_image' of a FixN4BiasFieldCorrection

Summary of what happened:

fMRIPrep failed on traceback, raising an exception while executing Node n4_correct.

Command used (and if a helper script was used, a link to the helper script or the command generated):

singularity run \
--cleanenv \
-B /shared/tonya/2021_fMRIMEG/BIDS_WORK/bids-raw:/data:ro \
-B /shared/tonya/2021_fMRIMEG/BIDS_WORK/.bids-raw_DB:/DB:ro \
-B /shared/tonya/2021_fMRIMEG/BIDS_WORK/derivatives:/derivs \
-B /scratch/tonya/work:/work \
-B /shared/tonya/2021_fMRIMEG/BIDS_WORK/derivatives/software/templateflow:/templateflow \
-B /scratch/tonya/work/freesurfer_license.txt:/work/freesurfer_license.txt \
/opt/SingularityImgs/fmriprep-23.2.3.sif \
/data /derivs/fmriprep participant \
--participant_label 03 \
-w /work/fmriprep \
--nprocs 4 \
--omp-nthreads 4 \
--mem 14 \
-v \
--no-sub \
--resource-monitor \
--skip_bids_validation 
--notrack \
--stop-on-first-crash \
--fs-license-file /work/freesurfer_license.txt

Version:

23.2.3

Environment (Docker, Singularity / Apptainer, custom installation):

Singularity

Data formatted according to a validatable standard? Please provide the output of the validator:

The data is a valid BIDS dataset. There are a few warnings, but they are mostly related to tsv and json files that I’ve yet to modify or other subjects (the subject being run was sub-03). The inconsistent parameters for sub-03 has a few less slices in the T1w sequence compared to other subjects, but no issues with the .nii.gz file itself.

Bids-validator Log for Subject: 03
Date: Mon 15 Jul 2024 16:33:25 ADT
------------------------------------------

bids-validator@1.14.6
	e[33m1: [WARN] Not all subjects contain the same files. Each subject should contain the same number of files with the same naming unless some files are known to be missing. (code: 38 - INCONSISTENT_SUBJECTS)e[39m
		./sub-10/func/sub-10_task-rest_bold.json
			Evidence: Subject: sub-10; Missing file: sub-10_task-rest_bold.json
		./sub-10/func/sub-10_task-rest_bold.nii.gz
			Evidence: Subject: sub-10; Missing file: sub-10_task-rest_bold.nii.gz
		./sub-10/func/sub-10_task-rest_events.tsv
			Evidence: Subject: sub-10; Missing file: sub-10_task-rest_events.tsv
		./sub-11/func/sub-11_task-rest_sbref.json
			Evidence: Subject: sub-11; Missing file: sub-11_task-rest_sbref.json
		./sub-11/func/sub-11_task-rest_sbref.nii.gz
			Evidence: Subject: sub-11; Missing file: sub-11_task-rest_sbref.nii.gz

e[36m	Please visit https://neurostars.org/search?q=INCONSISTENT_SUBJECTS for existing conversations about this issue.e[39m

	e[33m2: [WARN] Not all subjects/sessions/runs have the same scanning parameters. (code: 39 - INCONSISTENT_PARAMETERS)e[39m
		./sub-03/anat/sub-03_T1w.nii.gz
		./sub-03/anat/sub-03_mod-T1w_defacemask.nii.gz
		./sub-06/anat/sub-06_T1w.nii.gz
		./sub-06/anat/sub-06_mod-T1w_defacemask.nii.gz
		./sub-07/fmap/sub-07_dir-PA_epi.nii.gz
		./sub-10/anat/sub-10_T1w.nii.gz
		./sub-10/anat/sub-10_mod-T1w_defacemask.nii.gz
		./sub-11/anat/sub-11_T1w.nii.gz
		./sub-11/anat/sub-11_mod-T1w_defacemask.nii.gz
		./sub-11/func/sub-11_task-language_bold.nii.gz
		e[33m... and 5 more files having this issue (Use --verbose to see them all).e[39m

e[36m	Please visit https://neurostars.org/search?q=INCONSISTENT_PARAMETERS for existing conversations about this issue.e[39m

	e[33m3: [WARN] Tabular file contains custom columns not described in a data dictionary (code: 82 - CUSTOM_COLUMN_WITHOUT_DESCRIPTION)e[39m
		./sub-03/func/sub-03_task-language_events.tsv
			Evidence: Columns: TODO -- fill in rows and add more tab-separated columns if desired not defined, please define in: /events.json, /task-language_events.json,/sub-03/sub-03_events.json,/sub-03/sub-03_task-language_events.json,/sub-03/func/sub-03_events.json,/sub-03/func/sub-03_task-language_events.json
		./sub-03/func/sub-03_task-memory_events.tsv
			Evidence: Columns: TODO -- fill in rows and add more tab-separated columns if desired not defined, please define in: /events.json, /task-memory_events.json,/sub-03/sub-03_events.json,/sub-03/sub-03_task-memory_events.json,/sub-03/func/sub-03_events.json,/sub-03/func/sub-03_task-memory_events.json
		./sub-03/func/sub-03_task-rest_events.tsv
			Evidence: Columns: TODO -- fill in rows and add more tab-separated columns if desired not defined, please define in: /events.json, /task-rest_events.json,/sub-03/sub-03_events.json,/sub-03/sub-03_task-rest_events.json,/sub-03/func/sub-03_events.json,/sub-03/func/sub-03_task-rest_events.json
		./sub-04/func/sub-04_task-language_events.tsv
			Evidence: Columns: TODO -- fill in rows and add more tab-separated columns if desired not defined, please define in: /events.json, /task-language_events.json,/sub-04/sub-04_events.json,/sub-04/sub-04_task-language_events.json,/sub-04/func/sub-04_events.json,/sub-04/func/sub-04_task-language_events.json
		./sub-04/func/sub-04_task-memory_events.tsv
			Evidence: Columns: TODO -- fill in rows and add more tab-separated columns if desired not defined, please define in: /events.json, /task-memory_events.json,/sub-04/sub-04_events.json,/sub-04/sub-04_task-memory_events.json,/sub-04/func/sub-04_events.json,/sub-04/func/sub-04_task-memory_events.json
		./sub-04/func/sub-04_task-rest_events.tsv
			Evidence: Columns: TODO -- fill in rows and add more tab-separated columns if desired not defined, please define in: /events.json, /task-rest_events.json,/sub-04/sub-04_events.json,/sub-04/sub-04_task-rest_events.json,/sub-04/func/sub-04_events.json,/sub-04/func/sub-04_task-rest_events.json
		./sub-06/func/sub-06_task-language_events.tsv
			Evidence: Columns: TODO -- fill in rows and add more tab-separated columns if desired not defined, please define in: /events.json, /task-language_events.json,/sub-06/sub-06_events.json,/sub-06/sub-06_task-language_events.json,/sub-06/func/sub-06_events.json,/sub-06/func/sub-06_task-language_events.json
		./sub-06/func/sub-06_task-memory_events.tsv
			Evidence: Columns: TODO -- fill in rows and add more tab-separated columns if desired not defined, please define in: /events.json, /task-memory_events.json,/sub-06/sub-06_events.json,/sub-06/sub-06_task-memory_events.json,/sub-06/func/sub-06_events.json,/sub-06/func/sub-06_task-memory_events.json
		./sub-06/func/sub-06_task-rest_events.tsv
			Evidence: Columns: TODO -- fill in rows and add more tab-separated columns if desired not defined, please define in: /events.json, /task-rest_events.json,/sub-06/sub-06_events.json,/sub-06/sub-06_task-rest_events.json,/sub-06/func/sub-06_events.json,/sub-06/func/sub-06_task-rest_events.json
		./sub-07/func/sub-07_task-language_events.tsv
			Evidence: Columns: TODO -- fill in rows and add more tab-separated columns if desired not defined, please define in: /events.json, /task-language_events.json,/sub-07/sub-07_events.json,/sub-07/sub-07_task-language_events.json,/sub-07/func/sub-07_events.json,/sub-07/func/sub-07_task-language_events.json
		e[33m... and 16 more files having this issue (Use --verbose to see them all).e[39m

e[36m	Please visit https://neurostars.org/search?q=CUSTOM_COLUMN_WITHOUT_DESCRIPTION for existing conversations about this issue.e[39m

	e[33m4: [WARN] The recommended file /README is very small. Please consider expanding it with additional information about the dataset. (code: 213 - README_FILE_SMALL)e[39m
		./README

e[36m	Please visit https://neurostars.org/search?q=README_FILE_SMALL for existing conversations about this issue.e[39m

        e[34me[4mSummary:e[24me[39m                 e[34me[4mAvailable Tasks:e[24me[39m                         e[34me[4mAvailable Modalities:e[39me[24m 
        157 Files, 6.85GB        rest                                     MRI                   
        9 - Subjects             TODO: full task name for language                              
        1 - Session              TODO: full task name for memory                                
                                 TODO: full task name for rest                                  
                                 language                                                       
                                 memory                                                         


e[36m	If you have any questions, please post on https://neurostars.org/tags/bids.e[39m


Relevant log outputs (up to 20 lines):

Traceback (most recent call last):
  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node n4_correct.

Cmdline:
	N4BiasFieldCorrection --bspline-fitting [ 200 ] -d 3 --input-image /data/sub-03/func/sub-03_task-rest_sbref.nii.gz --output sub-03_task-rest_sbref_corrected.nii.gz -r --weight-image /work/fmriprep/fmriprep_23_2_wf/sub_03_wf/bold_task_rest_wf/bold_fit_wf/enhance_and_skullstrip_bold_wf/fix_header/tpl-MNI152NLin2009cAsym_res-01_label-brain_probseg_trans_fixhdr.nii.gz
Stdout:

Stderr:

Traceback:
	Traceback (most recent call last):
	  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 453, in aggregate_outputs
	    setattr(outputs, key, val)
	  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/interfaces/base/traits_extension.py", line 330, in validate
	    value = super(File, self).validate(objekt, name, value, return_pathlike=True)
	  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/interfaces/base/traits_extension.py", line 135, in validate
	    self.error(objekt, name, str(value))
	  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/traits/base_trait_handler.py", line 74, in error
	    raise TraitError(
	traits.trait_errors.TraitError: The 'output_image' trait of a _FixN4BiasFieldCorrectionOutputSpec instance must be a pathlike object or string representing an existing file, but a value of '/work/fmriprep/fmriprep_23_2_wf/sub_03_wf/bold_task_rest_wf/bold_fit_wf/enhance_and_skullstrip_bold_wf/n4_correct/sub-03_task-rest_sbref_corrected.nii.gz' <class 'str'> was specified.

	During handling of the above exception, another exception occurred:

	Traceback (most recent call last):
	  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 398, in run
	    runtime = self._post_run_hook(runtime)
	  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/interfaces/mixins/fixheader.py", line 127, in _post_run_hook
	    outputs = self.aggregate_outputs(runtime=runtime).get_traitsfree()
	  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 460, in aggregate_outputs
	    raise FileNotFoundError(msg)
	FileNotFoundError: No such file or directory '/work/fmriprep/fmriprep_23_2_wf/sub_03_wf/bold_task_rest_wf/bold_fit_wf/enhance_and_skullstrip_bold_wf/n4_correct/sub-03_task-rest_sbref_corrected.nii.gz' for output 'output_image' of a FixN4BiasFieldCorrection interface

Screenshots / relevant information:

I attempted several solutions, including increasing the allocated memory, and running the script with both version 24.0.0 and previous version 23.2.3. Both versions encountered the same error, and since my analyses are relatively simple (no DTI, just two event-based fMRI, rfMRI with one sbref, and a T1w for registration), either version will likely work for me.

Any help is greatly appreciated! Thank you!


Hi @icormier0021 and welcome to neurostars!

You’re currently giving the job only 14 megabytes of memory. Try making it 14000.

Best,
Steven

1 Like

This is a bug we recently found with multi-volume sbrefs. It will be fixed in 24.0.1. Or you can test with nipreps/fmriprep:unstable.

1 Like

Thank you! I had modified a previous script from my lab manager, and it listed the --mem flag as being in GB, not MB. That’s what I get for not double-checking the usage arguments…

Thank you for letting me know. I will attempt to use the unstable version to test it out and see if it works.