Missing fieldmap outputs and anat for additional sessions

Software Support
1

Summary of what happened:

Hi everyone,

I’m running into issues with preprocessing multiple sessions for a single subject.

I have multiple sessions for a single subject run. Whenever I run fmriprep, the session 1 fmriprep outputs look right. However, when I use the same script for other sessions for the same subject, I get missing fmap and anat folders under the additional sessions, and have an overarching anat directory under the subject but not specified to the sessions (see below for structure). I collect fieldmaps and T1 for each session, and have checked my anat and fmap .nii.gz and .json files - they’re named correctly and the IntendedFor correctly lists for the bold files collected during that session. Not sure what’s the issue with why I’m not seeing fmap directory under the preprocessed fmriprep output folders! Please see below for my data structure:

sub-XX 
---anat
---ses-01
--------anat
--------fmap
--------func
---ses-02
--------anat
------------ sub-XX_ses-02_from-orig_to-T1w_mode-image_xfm.txt
--------func
------------ all preprocessed bold

Command used (and if a helper script was used, a link to the helper script or the command generated):

docker run --rm -it \
    --platform linux/amd64 \
    -v "$1:/data:ro" \
    -v "$2:/out" \
    -v "$3:/workdir" \
    -v "$4:/opt/freesurfer/license.txt:ro" \
    -e OMP_NUM_THREADS=1 \
    -e KMP_AFFINITY=disabled \
    nipreps/fmriprep:latest \
    /data /out participant \
    --fs-license-file /opt/freesurfer/license.txt \
    --output-spaces MNI152NLin2009cAsym:res-2 \
    --skip-bids-validation \
    --participant-label $5 \
    --level full \
    --me-output-echos \
    --stop-on-first-crash

Version:

fMRIPrep-25.0.0

Environment (Docker, Singularity / Apptainer, custom installation):

Docker. I’m running the command shared above in a python script. But, this is the only place that calls fmriprep.

Data formatted according to a validatable standard? Please provide the output of the validator:

I ran dcm2bids, so should be validated to standard.

Relevant log outputs (up to 20 lines):

There were no errors thrown. I’m pasting a bit from the log that may show fmap was ran.

250601-19:16:10,236 nipype.workflow INFO:
	 [Node] Setting-up "fmriprep_25_0_wf.sub_51_wf.bold_ses_02_task_intervention_run_1_echo_1_wf.bold_fit_wf.fmapref_buffer" in "/tmp/work/fmriprep_25_0_wf/sub_51_wf/bold_ses_02_task_intervention_run_1_echo_1_wf/bold_fit_wf/fmapref_buffer".
250601-19:16:10,238 nipype.workflow INFO:
	 [Node] Executing "fmapref_buffer" <nipype.interfaces.utility.wrappers.Function>
250601-19:16:10,239 nipype.workflow INFO:
	 [Node] Finished "fmapref_buffer", elapsed time 0.000362s.
250601-19:16:10,280 nipype.interface WARNING:
	 Changing /out/sub-51/ses-02/func/sub-51_ses-02_task-intervention_run-1_desc-hmc_boldref.nii.gz dtype from uint16 to float32
250601-19:16:10,608 nipype.workflow INFO:
	 [Node] Setting-up "fmriprep_25_0_wf.sub_51_wf.bold_ses_02_task_movie_run_1_echo_1_wf.bold_fit_wf.fmapref_buffer" in "/tmp/work/fmriprep_25_0_wf/sub_51_wf/bold_ses_02_task_movie_run_1_echo_1_wf/bold_fit_wf/fmapref_buffer".
250601-19:16:10,609 nipype.workflow INFO:
	 [Node] Executing "fmapref_buffer" <nipype.interfaces.utility.wrappers.Function>
250601-19:16:10,610 nipype.workflow INFO:
	 [Node] Finished "fmapref_buffer", elapsed time 0.000348s.
250601-19:16:10,863 nipype.interface WARNING:
	 Changing /out/sub-51/ses-02/func/sub-51_ses-02_task-movie_run-1_desc-hmc_boldref.nii.gz dtype from uint16 to float32
250601-19:16:11,233 nipype.workflow INFO:
	 [Node] Setting-up "fmriprep_25_0_wf.sub_51_wf.bold_ses_02_task_movie_run_2_echo_1_wf.bold_fit_wf.fmapref_buffer" in "/tmp/work/fmriprep_25_0_wf/sub_51_wf/bold_ses_02_task_movie_run_2_echo_1_wf/bold_fit_wf/fmapref_buffer".

Screenshots / relevant information:

2

Hi @cchang and welcome to neurostars!

In the future please use the Software Support post category and post template so we can better assist you. You can see I edited in for you this time. Feel free to edit your post to provide the missing information.

This is normal. Unless specified otherwise (e.g., with BIDS filter files) fMRIPrep creates an anatomical based on all sessions’ T1w. This goes in an overarching anat folder while individual session-wise transformations to the subject-level template are saved out in session folders.

Best,
Steven

3

Thanks Steven! I updated my original post - let me know if I can add any other information.

As for the anat, that makes a lot of sense. Does this mean I can use the session specific .txt files to register to the session specific T1 images?

4

Hi @cchang,

Thanks for the info!

In theory yes but I don’t see many cases in which that would be warranted. Keep in mind you would need to combine the MNI-AverageT1 and AverageT1-SessionT1 transforms. I also wouldn’t recommend doing additional registration after fmriprep, as it introduces interpolation.

In regards to the fmap question, I would need to see more about your data and IntendedFor setups to make sure each session has fmaps. But, if in the output htmls you see all bold images have SDC, then you should be good to go.

Best,
Steven

5

Gotcha!

In theory yes but I don’t see many cases in which that would be warranted. Keep in mind you would need to combine the MNI-AverageT1 and AverageT1-SessionT1 transforms. I also wouldn’t recommend doing additional registration after fmriprep, as it introduces interpolation.

Since the fmriprep preprocessed bold outputs look like they’re in epi space, I’m interested in registering it to standard space (specifically MNI 2mm that FSL uses). Would the following code I’m using introduce heavy interpolation even if I’m primarily using the transformation matrices, but changing out {sub}_{ses}_space-MNI152NLin2009cAsym_res-2_desc-preproc_T1w.nii.gz for MNI152_T1_2mm.nii.gz?

def transform_epi_to_mni(base_dir, subject, session, task, run="1"):
    sub = f"sub-{subject}"
    ses = f"ses-{session}"
    task_label = f"task-{task}_run-{run}"

    epi_input = f"{base_dir}/derivatives/tedana/{sub}/{ses}/{task_label}/desc-denoised_tissueRegressed_bold.nii.gz"
    #t1_ref = f"{base_dir}/derivatives/fmriprep/{sub}/{ses}/anat/{sub}_{ses}_desc-preproc_T1w.nii.gz"
    t1_ref = f"{base_dir}/derivatives/fmriprep/{sub}/ses-01/anat/{sub}_ses-01_desc-preproc_T1w.nii.gz"
    bold_to_t1 = f"{base_dir}/derivatives/fmriprep/{sub}/{ses}/func/{sub}_{ses}_{task_label}_from-boldref_to-T1w_mode-image_desc-coreg_xfm.txt"
    epi_in_t1 = f"{base_dir}/derivatives/standard_reg/{sub}/{ses}/desc-denoised_tissueRegressed_bold_T1w.nii.gz"

    #mni_ref = f"{base_dir}/derivatives/fmriprep/{sub}/{ses}/anat/{sub}_{ses}_space-MNI152NLin2009cAsym_res-2_desc-preproc_T1w.nii.gz"
    mni_ref = "/usr/local/fsl/data/standard/MNI152_T1_2mm.nii.gz"
    t1_to_mni = f"{base_dir}/derivatives/fmriprep/{sub}/ses-01/anat/{sub}_ses-01_from-T1w_to-MNI152NLin2009cAsym_mode-image_xfm.h5"
    epi_in_mni = f"{base_dir}/derivatives/standard_reg/{sub}/{ses}/desc-denoised_tissueRegressed_bold_MNI152NLin2009cAsym.nii.gz"

    os.makedirs(os.path.dirname(epi_in_mni), exist_ok=True)

    # Apply EPI to T1 transform
    cmd1 = [
        "antsApplyTransforms", "-d", "3", "-e", "3",
        "-i", epi_input,
        "-r", t1_ref,
        "-t", bold_to_t1,
        "-o", epi_in_t1
    ]
    run_command(cmd1)

    # Apply T1 to MNI transform
    cmd2 = [
        "antsApplyTransforms", "-d", "3", "-e", "3",
        "-i", epi_in_t1,
        "-r", mni_ref,
        "-t", t1_to_mni,
        "-o", epi_in_mni
    ]
    run_command(cmd2)

    # Validate with fslinfo
    ref_info = get_fslinfo(mni_ref)
    out_info = get_fslinfo(epi_in_mni)
    match, key, ref_val, out_val = compare_fslinfo(ref_info, out_info)

In regards to the fmap question, I would need to see more about your data and IntendedFor setups to make sure each session has fmaps.

This is the title of one of the fmaps from session-02: sub-51_ses-02_dir-AP_epi. I have both nii.gz and .json files with this file name. The intendedFor in this .json files (along with other .json files) is as follows (which includes all of the bold sessions I’ve collected):

“ConversionSoftware”: “dcm2niix”,
“ConversionSoftwareVersion”: “v1.0.20241211”,
“Dcm2bidsVersion”: “3.2.0”,
“IntendedFor”: [
“func/sub-51_ses-02_task-intervention_run-1_echo-3_bold.nii.gz”,
“func/sub-51_ses-02_task-movie_run-4_echo-3_bold.nii.gz”,
“func/sub-51_ses-02_task-rest_run-1_echo-1_bold.nii.gz”,
“func/sub-51_ses-02_task-movie_run-3_echo-2_bold.nii.gz”,
“func/sub-51_ses-02_task-movie_run-3_echo-1_bold.nii.gz”,
“func/sub-51_ses-02_task-rest_run-1_echo-2_bold.nii.gz”,
“func/sub-51_ses-02_task-movie_run-1_echo-3_bold.nii.gz”,
“func/sub-51_ses-02_task-movie_run-2_echo-3_bold.nii.gz”,
“func/sub-51_ses-02_task-movie_run-3_echo-3_bold.nii.gz”,
“func/sub-51_ses-02_task-movie_run-1_echo-1_bold.nii.gz”,
“func/sub-51_ses-02_task-movie_run-2_echo-1_bold.nii.gz”,
“func/sub-51_ses-02_task-movie_run-4_echo-2_bold.nii.gz”,
“func/sub-51_ses-02_task-intervention_run-1_echo-2_bold.nii.gz”,
“func/sub-51_ses-02_task-intervention_run-1_echo-1_bold.nii.gz”,
“func/sub-51_ses-02_task-movie_run-4_echo-1_bold.nii.gz”,
“func/sub-51_ses-02_task-movie_run-2_echo-2_bold.nii.gz”,
“func/sub-51_ses-02_task-movie_run-1_echo-2_bold.nii.gz”,
“func/sub-51_ses-02_task-rest_run-1_echo-3_bold.nii.gz”
]

But, if in the output htmls you see all bold images have SDC, then you should be good to go.
So, there’s only one .html per subject (can see compiled session runs within this .html if that sounds right to you). For some reason, for my .html summary for ses-01 it says SDC: None:

  • Original orientation: LAS
  • Repetition time (TR): 1.7s
  • Phase-encoding (PE) direction: Posterior-Anterior
  • Multi-echo EPI sequence: 3 echoes.
  • Slice timing correction: Applied
    Susceptibility distortion correction: None
  • Registration: FreeSurfer bbregister (boundary-based registration, BBR) - 6 dof
  • Non-steady-state volumes: 1

But, I see the SDC summary results for each bold image in session 1 files . Session 1 also has B0 field mapping results reported (Reports for: session 01, fmapid auto00000.). However, sessions 2 and 3 for the same subject don’t have SDC summary results (no fmapid auto00000 or individual SDC summary results for each bold image).

6

Hi @cchang,

You specified MNI as your output space. What makes you think they’re in EPI space? Also if you want to use the FSL version, it’s MNI152NLin6Asym.

Looks like SDC is not being performed. Your IntendedFor entries are missing the session folder.

Best,
Steven

7

Hi @Steven

So, if I fslhd my preprocessed bold image “Pilot/derivatives/fmriprep/sub-51/ses-01/func/sub-51_ses-01_task-rest_run-1_echo-1_desc-preproc_bold.nii.gz”, these are the output dimensions I get:
dim1 112
dim2 112
dim3 76
vox_units mm

I do see the preprocessed bold image registered to MNI152 (“Pilot/derivatives/fmriprep/sub-51/ses-01/func/sub-51_ses-01_task-rest_run-1_space-MNI152NLin2009cAsym_res-2_desc-preproc_bold.nii.gz”) but this doesn’t seem to have the individual echo outputs.

I see, so if I want the output to be 91 x 109 x 91 dimensions then I should set the output in my docker function to be MNI152NLin6Asym.

I see. Session 1 IntendedFor is also set up the same way and it’s running, so I didn’t realize that was the issue. To confirm, I should set up the IntendedFor to be the following for SDC to run. Is there anything else I need to toggle on?

“IntendedFor”: [
“ses-02/func/sub-51_ses-02_task-intervention_run-1_echo-3_bold.nii.gz”,
“ses-02/func/sub-51_ses-02_task-movie_run-4_echo-3_bold.nii.gz”,
“ses-02/func/sub-51_ses-02_task-rest_run-1_echo-1_bold.nii.gz”,
“ses-02/func/sub-51_ses-02_task-movie_run-3_echo-2_bold.nii.gz”,
“ses-02/func/sub-51_ses-02_task-movie_run-3_echo-1_bold.nii.gz”,
“ses-02/func/sub-51_ses-02_task-rest_run-1_echo-2_bold.nii.gz”…
]

8

Yes with res-02. You can see the resolutions here: tpl-MNI152NLin6Asym/template_description.json at a68885d55e5669fd86b8c749ec97550d8fb5949d · templateflow/tpl-MNI152NLin6Asym · GitHub

I am not a multi-echo user, it is possible that fmriprep only outputs the combined file in the final output space, but I can’t confirm.

Looks right to me.

Best,
Steven

9

Thanks for all of your detailed responses @Steven ! I’ll do a more detailed review of the multi-echo output.

Quick question on the preprocessing time. I’m running on docker with 32 GB Memory Limit and 12 CPU limit. My fmriprep pipeline takes 4 hours to run per subject session. Does this timing sound right to you?

10

Hi @cchang,

Do you have different jobs for sessions within subjects? fMRIPrep parallelizes that for you. 4 hours seems short for a single BOLD run, let alone the multiple files you have. recon-all itself can take about that long.

Best,
Steven

11

I do not. I run fmriprep separately for each session (each with a different docker image). What’s the general timeframe it takes to run fmriprep?

@Steven quick update on the previous issue: I’ve updated my IntendedFor files to the ones shared above and updated my reference image to MNI152NLin6Asym:res-2 (shown below). But, I’m getting the following error about 1.5 hours into the fmriprep process. I’ve read that it’s a memory issue, but I seem to have a lot of memory left and have never come across this issue when I ran session 01 for the same subject:

docker run --rm -it \
    --platform linux/amd64 \
    -v "$1:/data:ro" \
    -v "$2:/out" \
    -v "$3:/workdir" \
    -v "$4:/opt/freesurfer/license.txt:ro" \
    -e OMP_NUM_THREADS=1 \
    -e KMP_AFFINITY=disabled \
    nipreps/fmriprep:latest \
    /data /out participant \
    --fs-license-file /opt/freesurfer/license.txt \
    --output-spaces MNI152NLin6Asym:res-2 \
    --skip-bids-validation \
    --participant-label $5 \
    --level full \
    --me-output-echos \
    --stop-on-first-crash

Error that I’m getting from fmriprep

Cmdline:
	t2smap -d /tmp/work/fmriprep_25_0_wf/sub_51_wf/bold_ses_02_task_movie_run_1_echo_1_wf/bold_native_wf/_echoidx_0/boldref_bold/sub-51_ses-02_task-movie_run-1_echo-1_bold_tshift_xformresampled.nii.gz /tmp/work/fmriprep_25_0_wf/sub_51_wf/bold_ses_02_task_movie_run_1_echo_1_wf/bold_native_wf/_echoidx_1/boldref_bold/sub-51_ses-02_task-movie_run-1_echo-2_bold_tshift_xformresampled.nii.gz /tmp/work/fmriprep_25_0_wf/sub_51_wf/bold_ses_02_task_movie_run_1_echo_1_wf/bold_native_wf/_echoidx_2/boldref_bold/sub-51_ses-02_task-movie_run-1_echo-3_bold_tshift_xformresampled.nii.gz -e 16.400000000000002 37.6 58.78 --mask /tmp/work/fmriprep_25_0_wf/sub_51_wf/bold_ses_02_task_movie_run_1_echo_1_wf/bold_native_wf/bold_t2smap_wf/dilate_mask/dilated_mask.nii.gz --fittype curvefit
Stdout:

Stderr:
	INFO     t2smap:t2smap_workflow:233 Using output directory: /tmp/work/fmriprep_25_0_wf/sub_51_wf/bold_ses_02_task_movie_run_1_echo_1_wf/bold_native_wf/bold_t2smap_wf/t2smap_node
	INFO     t2smap:t2smap_workflow:243 Loading input data: ['/tmp/work/fmriprep_25_0_wf/sub_51_wf/bold_ses_02_task_movie_run_1_echo_1_wf/bold_native_wf/_echoidx_0/boldref_bold/sub-51_ses-02_task-movie_run-1_echo-1_bold_tshift_xformresampled.nii.gz', '/tmp/work/fmriprep_25_0_wf/sub_51_wf/bold_ses_02_task_movie_run_1_echo_1_wf/bold_native_wf/_echoidx_1/boldref_bold/sub-51_ses-02_task-movie_run-1_echo-2_bold_tshift_xformresampled.nii.gz', '/tmp/work/fmriprep_25_0_wf/sub_51_wf/bold_ses_02_task_movie_run_1_echo_1_wf/bold_native_wf/_echoidx_2/boldref_bold/sub-51_ses-02_task-movie_run-1_echo-3_bold_tshift_xformresampled.nii.gz']
	INFO     t2smap:t2smap_workflow:259 Using user-defined mask
	INFO     t2smap:t2smap_workflow:262 Computing adaptive T2* map
	INFO     t2smap:t2smap_workflow:279 Computing optimal combination
	INFO     combine:make_optcom:191 Optimally combining data with voxel-wise T2* estimates
	Killed
Traceback:
	Traceback (most recent call last):
	  File "/opt/conda/envs/fmriprep/lib/python3.12/site-packages/nipype/interfaces/base/core.py", line 457, in aggregate_outputs
	    setattr(outputs, key, val)
	  File "/opt/conda/envs/fmriprep/lib/python3.12/site-packages/nipype/interfaces/base/traits_extension.py", line 325, in validate
	    value = super().validate(objekt, name, value, return_pathlike=True)
	            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
	  File "/opt/conda/envs/fmriprep/lib/python3.12/site-packages/nipype/interfaces/base/traits_extension.py", line 135, in validate
	    self.error(objekt, name, str(value))
	  File "/opt/conda/envs/fmriprep/lib/python3.12/site-packages/traits/base_trait_handler.py", line 74, in error
	    raise TraitError(
	traits.trait_errors.TraitError: The 't2star_map' trait of a T2SMapOutputSpec instance must be a pathlike object or string representing an existing file, but a value of '/tmp/work/fmriprep_25_0_wf/sub_51_wf/bold_ses_02_task_movie_run_1_echo_1_wf/bold_native_wf/bold_t2smap_wf/t2smap_node/T2starmap.nii.gz' <class 'str'> was specified.

	During handling of the above exception, another exception occurred:

	Traceback (most recent call last):
	  File "/opt/conda/envs/fmriprep/lib/python3.12/site-packages/nipype/interfaces/base/core.py", line 404, in run
	    outputs = self.aggregate_outputs(runtime)
	              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
	  File "/opt/conda/envs/fmriprep/lib/python3.12/site-packages/nipype/interfaces/base/core.py", line 464, in aggregate_outputs
	    raise FileNotFoundError(msg)
	FileNotFoundError: No such file or directory '/tmp/work/fmriprep_25_0_wf/sub_51_wf/bold_ses_02_task_movie_run_1_echo_1_wf/bold_native_wf/bold_t2smap_wf/t2smap_node/T2starmap.nii.gz' for output 't2star_map' of a T2SMap interface
12

Hi @cchang,

I don’t see where that is happening, as there’s no BIDS filter files specified. Unless there’s something happening before the docker run command I am not seeing. In either case, I would recommend not separating sessions, unless there is some specific reason particular to your analysis to do so.

This is also going to bump the run time way higher, compared to using multithreaded processes.

Have you confirmed in your Docker settings that Docker has access to enough memory? It doesn’t use the whole system memory by default. Have you looked at any resource monitoring during the process?

It would also be nice to get the BIDS validator output, to make sure the dataset is ready to go.

Best,
Steven

13

@Steven

I see - why wouldn’t you recommend separating the sessions? There’s a python code that integrates the bids2nixx command and the fmriprep docker command which specifies the session to run since each session is collected separately and a few weeks apart. So, I’ve been preprocessing the data as they’ve been collected.

I will remove OMP_NUM_THREADS=1

My Docker is set at 12 CPU limit, 32 GB Memory Limit, 1.5 BF Swap limit, and 632 BG Disk Usage limit (my computer is 4TB). I haven’t looked at any resources monitoring during the process. Is this under Activity Monitor? Is the error message thrown a result of memory limits?

I’ll double check this!

14

Hi @cchang,

Because then each session has it’s own anatomical template, as opposed to using a consistent subject-average template. It can have some legitimate use cases such as infant brains that develop quickly. But adults separated by a few weeks shouldn’t have drastically different brains, so aligning to a single average might lead to better session-to-session alignment for longitudinal analyses.

I think fMRIprep has a built in resource monitor argument. I don’t know what Activity Monitor is referring to. I know there’s something on Macs called Activity Monitor, but you specified a linux distribution in your docker run command, so I do not know what the Linux alternative is.

Yes.

Best,
Steven

15

I see! This is super helpful context, so I really appreciate it. It turns out my docker bash command doesn’t actually take in any session specific information, so I’m preprocessing all sessions for a subject together. Sounds like no changes need to be made here!

I’m curious, since I’m rerunning the docker command when it crashes, would you recommend that I clear out my “work” directory? fmriprep seems to be overwriting my func files everytime, but my anat files stay untouched. Is a common observation?

I’m actually running fmri on a docker application on my Mac since it has an M2 chip (hence the “arm” specification in my docker command). Do you have any recommendations to track resources given this setup?

One last question about data structure/organization! My fmriprep outputs are stored separately in a derivatives folder under the root folder and not under the subject specific folder that holds the bids compliant unprocessed data. Is this data structure compliant with how fmriprep outputs are usually stored?

Pilot/
├── sub-01/
│   ├── ses-01/
│   │   ├── anat/
│   │   │   └── sub-01_ses-01_T1w.nii.gz
│   │   └── func/
│   │       ├── sub-01_ses-01_task-rest_bold.nii.gz
│   │       └── sub-01_ses-01_task-movie_bold.nii.gz
│   │   └── fmap/
│   │       ├── sub-01_ses-01_AP_fmap.nii.gz
│   │       └── sub-01_ses-01_PA_fmap.nii.gz
├── derivatives/
│     │ └── fmriprep/
│     │     └── sub-01/
│     │         └── ses-01/
│     │             └── func/
│     │                  └── sub-01_ses-01_task-rest_bold_preproc.nii.gz
│     │        └── ses-02/
│     │             └── func/
│     │                 └── sub-01_ses-02_task-rest_bold_preproc.nii.gz
16

Hi @cchang,

These names are not BIDS valid. They should be dir-AP/dir-PA. Also, if you are doing reverse phase encoding fieldmaps / PEPOLAR, they should have the epi suffix. See more here: Magnetic Resonance Imaging - Brain Imaging Data Structure 1.10.0

Worth a shot.

I don’t see any arm specification. I see a linux/amd64, and amd64 usually refers to Intel chips to my understanding, not Apple Silicon.

Yes, the subject folders in the BIDS root directory should only contain raw data. Derived data goes in derivatives.

Best,
Steven

17

Awesome - thank you so much! Latest update: got everything to run so really appreciate your input and help! @Steven

1 Like