fMRIPrep 25.2.5 reports 'finished successfully' but produces no desc-preproc_bold or confounds files

Software Support
1

Summary of what happened:

Running fMRIPrep 25.2.5 via Singularity with --level full (the default, not explicitly set at first, then confirmed explicitly). The run completes with “fMRIPrep finished successfully!” printed in the log and a 0-looking exit, but the func/ output directory only contains early registration/motion-correction files – no *_desc-preproc_bold.nii.gz, no *_desc-confounds_timeseries.tsv, for any task or participant. Reproduced identically across 2 of 9 participants so far (others still running).

What the resolved config (fmriprep.toml) confirms:

level = "full"
spaces = "T1w MNI152NLin2009cAsym"
output_layout = "bids"

What actually exists in func/ output (for each of 2 tasks per participant):

  • *_desc-brain_mask.nii.gz / .json
  • *_desc-coreg_boldref.nii.gz / .json
  • *_desc-hmc_boldref.nii.gz / .json
  • *_from-orig_to-boldref_mode-image_desc-hmc_xfm.txt / .json

What’s missing: any final resampled BOLD (desc-preproc_bold, any space), any confounds file, any carpet plot.

Has anyone seen this specific “fit-only, no apply/derivatives” behavior with 25.2.5? Is there an additional flag needed in this version to trigger the resampling/confounds stage within a single invocation, given the fit/derivatives workflow split introduced in 23.2? Happy to provide the full .out log or fmriprep.toml if useful.

Command used (and if a helper script was used, a link to the helper script or the command generated):

singularity run --cleanenv \
    -B ${STUDY}:/data \
    -B ${WORK}:/scratch_work \
    -B /work/afslab:/work/afslab \
    fmriprep_25.2.5.sif \
    /data/rawdata /data/derivatives/fmriprep participant \
    --participant-label ${SUBJECT} \
    --fs-license-file /work/afslab/freesurfer/license.txt \
    --work-dir /scratch_work \
    --output-spaces T1w \
    --nprocs 8

Version:

25.2.5

Environment (Docker, Singularity / Apptainer, custom installation):

Singularity

Data formatted according to a validatable standard? Please provide the output of the validator:

PASTE VALIDATOR OUTPUT HERE

Relevant log outputs (up to 20 lines):

What the generated CITATION.md boilerplate says (seemingly describing full processing, which doesn’t match the actual output):

“For each of the 2 BOLD runs found per subject… All resamplings can be performed with a single interpolation step by composing all the pertinent transformations… Gridded (volumetric) resamplings were performed using nitransforms, configured with cubic B-spline interpolation.”

Screenshots / relevant information:

  • BIDS layout summary in the config log reads Subjects: 1 | Sessions: 0 | Runs: 0 for this participant (our filenames don’t use a run- BIDS entity since each task occurs once per session – not sure if this is related)
  • No errors or warnings in the .err log beyond two benign ones (SINGULARITYENV_TEMPLATEFLOW_HOME/APPTAINERENV preference notice, and a Node.js “closing directory handle on garbage collection” warning)
  • Container is GACRC’s (Georgia Advanced Computing Resource Center) centrally-installed image at /apps/singularity-images/fmriprep_25.2.5.sif
  • FreeSurfer license and TemplateFlow cache both confirmed present and correctly bind-mounted
2

Hi @Ashley_Sanders and welcome to neurostars!

May you please add in the portion of the post that asks for the BIDS validator report? Those outputs definitely seem like the --level minimal output set, which is odd. Just to be sure, have you tried rerunning with a fresh working directory? And if you do try, can you try to add these flags:

--level full \
--write-graph \
-vvv \

I am curious if the workflow graph that is produced actually has all the full processing steps plotted out.

Best,
Steven

3

Update: found the root cause for those two subjects, and it’s resolved. I apologize for my delayed response.

Turned out to be a nipype thread-allocation deadlock. fMRIPrep was auto-deriving omp_nthreads=7 from --nprocs 8 (never set explicitly). With 2 BOLD runs per participant, two nodes each trying to claim 7 of the 8 available threads could never both be scheduled (confirmed via repeating “Cannot allocate job… 7 threads” messages in a -vvv diagnostic run that never progressed past that point). This also explains why it was intermittent across participants rather than universal….whether the deadlock actually triggers seems to depend on nipype’s internal scheduling timing for that particular run, not a deterministic per-participant difference.

Fix: explicit --omp-nthreads 2. Reran the two subjects with this added, and both now complete successfully with full expected output (desc-preproc_bold.nii.gz and desc-confounds_timeseries.tsv for both tasks). Added the flag to the real batch script for any future runs.

Thanks for the diagnostic suggestions (explicit --level full, --write-graph, -vvv, fresh work dir). That combination is what actually surfaced the thread-allocation messages clearly enough to pin this down. Happy to share the full diagnostic log if it’d be useful for anyone else who runs into similar intermittent fMRIPrep failures with multi-run participants.

2 Likes