Summary of what happened:
1: Ran fMRIPrep to produce surface (gifti, fsaverage5) and volume (nii.gz, MNI) outputs.
2: Kept all directories and files (fmriprep_25_2_wf, derivatives, BIDS) in place; made a copy of the derivatives subdirectory.
3: Added the bold, sbref, and fmaps for one additional run to the BIDS (input dataset) directory and ran fMRIPrep again with the same command as before (all previous files still present). The new run had its own pair fieldmaps (with their own unique B0 name); its fieldmaps were not used elsewhere.
4: Examined and compared derivatives and reports from before and after the second fmriprep job.
Since the previous freesurfer and working files were present, I expected fmriprep to reuse them, and only process the new run’s files. The second job finished much, much faster than the first, and as far as I can tell freesurfer wasn’t rerun. I was surprised, though, that nearly all of the derivatives had new timestamps; I expected files for the new run to be added to the existing func and fmap derivatives directory, not for (almost) all of the previous files to be replaced. (“Almost” since the surface anat derivatives appear unchanged.)
Question: Should I have been surprised? (Hunting through the documentation, I saw a few mentions that many pipeline components always run, even when unchanged, which implies I should expect the “normal” amount of fmriprep variability in output.)
Comparing the visual reports and outputs, most were reasonably similar (the normal few decimal places different) … except for one run. For example, the DVARS & FD were much bumpier in the first set of derivatives than the second:
Perhaps the sub-TD_ses-TD7T1_task-rest_run-10_desc-hmc_boldref.json file is a hint of what happened:
first set of derivatives:
{ "Sources": [ "bids:raw:sub-TD/ses-TD7T1/func/sub-TD_ses-TD7T1_task-rest_run-10_echo-1_bold.nii.gz" ] }
second:
{ "Sources": [ "bids:raw:sub-TD/ses-TD7T1/func/sub-TD_ses-TD7T1_task-rest_run-10_echo-2_bold.nii.gz" ] }
run-10 is multi-echo, but only two echoes, so the T2starmap wasn’t created. Maybe an echo is chosen at random for use in some steps? The sub-TD_ses-TD7T1_task-rest_run-10_space-MNI152NLin2009cAsym_desc-preproc_bold.json files are identical for the two jobs, both have “bids:raw:sub-TD/ses-TD7T1/func/sub-TD_ses-TD7T1_task-rest_run-10_echo-2_bold.nii.gz” as the first Sources. The preprocessed images are not catastrophically different at first look, but more variable (between the two fmriprep jobs) than the other runs.
Thanks!
Command used (and if a helper script was used, a link to the helper script or the command generated):
/app/.pixi/envs/fmriprep/bin/fmriprep --fs-license-file /tmp/.license/freesurfer/license.txt -w /tmp /tmp/BIDS /tmp/derivatives participant --participant_label TD --output-spaces fsaverage5 MNI152NLin2009cAsym --n-cpus 16 --omp-nthreads 4 --mem-mb 64000 -v
Version:
fMRIPrep version 25.2.2, FreeSurfer 7.3.2
Environment (Docker, Singularity / Apptainer, custom installation):
singularity
Data formatted according to a validatable standard? Please provide the output of the validator:
first fmriprep job:
You are using fMRIPrep-25.2.2, and a newer version of fMRIPrep is available: 25.2.3.
Please check out our documentation about how and when to upgrade:
https://fmriprep.readthedocs.io/en/latest/faq.html#upgrading
251106-09:59:07,528 cli INFO:
Telemetry system to collect crashes and errors is enabled - thanks for your feedback!. Use option ``--notrack`` to opt out.
251106-09:59:16,469 cli INFO:
Making sure the input data is BIDS compliant (warnings can be ignored in most cases).
bids-validator@1.14.10
(node:39558) Warning: Closing directory handle on garbage collection
(Use `node --trace-warnings ...` to show where the warning was created)
e[32mThis dataset appears to be BIDS compatible.e[39m
e[34me[4mSummary:e[24me[39m e[34me[4mAvailable Tasks:e[24me[39m e[34me[4mAvailable Modalities:e[39me[24m
62 Files, 3.58GB rest MRI
1 - Subject
1 - Session
...
251107-13:21:49,238 nipype.workflow IMPORTANT:
Works derived from this fMRIPrep execution should include the boilerplate text found in <OUTPUT_PATH>/logs/CITATION.md.
second fmriprep job (same directories as above, plus BIDS bold & sbref, fmaps for one additional run):
You are using fMRIPrep-25.2.2, and a newer version of fMRIPrep is available: 25.2.3.
Please check out our documentation about how and when to upgrade:
https://fmriprep.readthedocs.io/en/latest/faq.html#upgrading
251120-03:48:12,746 cli INFO:
Telemetry system to collect crashes and errors is enabled - thanks for your feedback!. Use option ``--notrack`` to opt out.
251120-03:48:22,366 cli INFO:
Making sure the input data is BIDS compliant (warnings can be ignored in most cases).
bids-validator@1.14.10
(node:11003) Warning: Closing directory handle on garbage collection
(Use `node --trace-warnings ...` to show where the warning was created)
e[32mThis dataset appears to be BIDS compatible.e[39m
e[34me[4mSummary:e[24me[39m e[34me[4mAvailable Tasks:e[24me[39m e[34me[4mAvailable Modalities:e[39me[24m
70 Files, 3.65GB rest MRI
1 - Subject
1 - Session
...
251120-04:56:44,99 nipype.workflow IMPORTANT:
Works derived from this fMRIPrep execution should include the boilerplate text found in <OUTPUT_PATH>/logs/CITATION.md.
Relevant log outputs (up to 20 lines):
PASTE LOG OUTPUT HERE