Fmaps Inclusion in the Config File Prevents fMRIPrep to Create the func folder

AJavan · March 31, 2023, 1:42pm

Summary of what happened:

Hello,

I am running fMRIprep on a resting state fMRI dataset where each subject has multiple sessions. To define the session for fMRIPrep, I have created a config.json file including the session name (see below). Initially, I included only anat and func as the datatypes in the config file and did not include the fmap (I was not aware that if a config file is provided, only datatypes specified there will be processed). Accordingly, the distortion correction was not applied but every other step has been fully applied and all the output folders(anat/fmap/func)/files were created and the html report and QC looked good.

To ensure that the distortion correction is included and that fMRIPrep has access to the fmaps, I added fmap to the config file and ran fMRIPrep again, this time it was completed but it did not create the func folder.

Here is the structure of the bids source directory for a subject and a session:

ses-FU48A
├── anat
│   ├── sub-MTL0003_ses-FU48A_FLAIR.json
│   ├── sub-MTL0003_ses-FU48A_FLAIR.nii.gz
│   ├── sub-MTL0003_ses-FU48A_run-01_T1w.json
│   ├── sub-MTL0003_ses-FU48A_run-01_T1w.nii.gz
│   ├── sub-MTL0003_ses-FU48A_run-02_T1w.json
│   ├── sub-MTL0003_ses-FU48A_run-02_T1w.nii.gz
│   ├── sub-MTL0003_ses-FU48A_T2star.json
│   └── sub-MTL0003_ses-FU48A_T2star.nii.gz
├── asl
│   ├── sub-MTL0003_ses-FU48A_asl.json
│   └── sub-MTL0003_ses-FU48A_asl.nii.gz
├── dwi
│   ├── sub-MTL0003_ses-FU48A_dwi.bval
│   ├── sub-MTL0003_ses-FU48A_dwi.bvec
│   ├── sub-MTL0003_ses-FU48A_dwi.json
│   └── sub-MTL0003_ses-FU48A_dwi.nii.gz
├── fmap
│   ├── sub-MTL0003_ses-FU48A_run-01_magnitude1.json
│   ├── sub-MTL0003_ses-FU48A_run-01_magnitude1.nii.gz
│   ├── sub-MTL0003_ses-FU48A_run-01_magnitude2.json
│   ├── sub-MTL0003_ses-FU48A_run-01_magnitude2.nii.gz
│   ├── sub-MTL0003_ses-FU48A_run-01_phasediff.json
│   ├── sub-MTL0003_ses-FU48A_run-01_phasediff.nii.gz
│   ├── sub-MTL0003_ses-FU48A_run-02_magnitude1.json
│   ├── sub-MTL0003_ses-FU48A_run-02_magnitude1.nii.gz
│   ├── sub-MTL0003_ses-FU48A_run-02_magnitude2.json
│   ├── sub-MTL0003_ses-FU48A_run-02_magnitude2.nii.gz
│   ├── sub-MTL0003_ses-FU48A_run-02_phasediff.json
│   ├── sub-MTL0003_ses-FU48A_run-02_phasediff.nii.gz
│   ├── sub-MTL0003_ses-FU48A_run-03_magnitude1.json
│   ├── sub-MTL0003_ses-FU48A_run-03_magnitude1.nii.gz
│   ├── sub-MTL0003_ses-FU48A_run-03_magnitude2.json
│   ├── sub-MTL0003_ses-FU48A_run-03_magnitude2.nii.gz
│   ├── sub-MTL0003_ses-FU48A_run-03_phasediff.json
│   ├── sub-MTL0003_ses-FU48A_run-03_phasediff.nii.gz
│   ├── sub-MTL0003_ses-FU48A_run-04_magnitude1.json
│   ├── sub-MTL0003_ses-FU48A_run-04_magnitude1.nii.gz
│   ├── sub-MTL0003_ses-FU48A_run-04_magnitude2.json
│   ├── sub-MTL0003_ses-FU48A_run-04_magnitude2.nii.gz
│   ├── sub-MTL0003_ses-FU48A_run-04_phasediff.json
│   ├── sub-MTL0003_ses-FU48A_run-04_phasediff.nii.gz
│   ├── sub-MTL0003_ses-FU48A_run-05_magnitude1.json
│   ├── sub-MTL0003_ses-FU48A_run-05_magnitude1.nii.gz
│   ├── sub-MTL0003_ses-FU48A_run-05_magnitude2.json
│   ├── sub-MTL0003_ses-FU48A_run-05_magnitude2.nii.gz
│   ├── sub-MTL0003_ses-FU48A_run-05_phasediff.json
│   └── sub-MTL0003_ses-FU48A_run-05_phasediff.nii.gz
└── func
    ├── sub-MTL0003_ses-FU48A_task-enc_bold.json
    ├── sub-MTL0003_ses-FU48A_task-enc_bold.nii.gz
    ├── sub-MTL0003_ses-FU48A_task-enc_events.edat2
    ├── sub-MTL0003_ses-FU48A_task-enc_events.txt
    ├── sub-MTL0003_ses-FU48A_task-rest_run-1_bold.json
    ├── sub-MTL0003_ses-FU48A_task-rest_run-1_bold.nii.gz
    ├── sub-MTL0003_ses-FU48A_task-rest_run-2_bold.json
    ├── sub-MTL0003_ses-FU48A_task-rest_run-2_bold.nii.gz
    ├── sub-MTL0003_ses-FU48A_task-ret_bold.json
    ├── sub-MTL0003_ses-FU48A_task-ret_bold.nii.gz
    ├── sub-MTL0003_ses-FU48A_task-ret_events.edat2
    └── sub-MTL0003_ses-FU48A_task-ret_events.txt

Command used (and if a helper script was used, a link to the helper script or the command generated):

#!/bin/bash
#SBATCH -J prep-MTL0003
#SBATCH --time=13:00:00
#SBATCH -n 1
#SBATCH --account=ctb-villens
#SBATCH --cpus-per-task=3
#SBATCH --mem-per-cpu=4G
#SBATCH --mail-type=None

# Load singularity module
SINGULARITY_IMG="/project/ctb-villens/quarantine/Core/fmriprep/22.0.0/fmriprep-22.0.0.simg"
module load VilleneuveLab
module load singularity/3.8

# Allocate your input and output directories, and allocate local scratch as the working directory
WORK_DIR=$(mktemp -d -p "${SLURM_TMPDIR}" work_dir-XXXX) || exit 1
echo "working directory is $work_dir"/home/javan/scratch/multi_prep/working_dir

# Define path for TSV file
STUDY="/home/javan/scratch/multi_prep/final/scripts/fmriprep/part1"

# Designate the location of your templateflow templates
export SINGULARITYENV_TEMPLATEFLOW_HOME=/project/ctb-villens/javan/templateflow
TEMPLATEFLOW_HOME="/project/ctb-villens/javan/templateflow"
export SINGULARITYENV_FS_LICENSE=/project/ctb-villens/quarantine/Core/freesurfer/7.1.0/license.txt
FS_LICENSE="/project/ctb-villens/quarantine/Core/freesurfer/7.1.0/license.txt"

#Parse the participants.tsv file and extract one subject ID from the line corresponding to this SLURM task
# The $((${SLURM_ARRAY_TASK_ID} + 1)) section is responsible for scipping the first line of the  participant tsv file

#subject=$( sed -n -E "$((${SLURM_ARRAY_TASK_ID} + 1))s/sub-(\S*)\>.*/\1/gp" ${STUDY}/participant.tsv)
subject=MTL0003
# Build your command to tell singularity to run fmriprep, ensuring it binds to your template directory, your scratch folder
SINGULARITY_CMD="singularity run --cleanenv \
 -B ${TEMPLATEFLOW_HOME}:${SINGULARITYENV_TEMPLATEFLOW_HOME} \
 -B ${FS_LICENSE}:${SINGULARITYENV_FS_LICENSE} \
 -B /project/ctb-villens/dataset/PreventAD/mri/sourcedata/bids:/source_bids:ro \
 -B /home/javan/scratch/distortion_correction/derivative_exfreesurfer:/out \
 -B /home/javan/scratch/prep_all_part2/script/final/code_final/${subject}:/jbid \
 -B "${WORK_DIR}":/work \
 ${SINGULARITY_IMG} \
 /source_bids /out \
 participant --participant-label ${subject} \
 -w /work -vv --nthreads ${SLURM_CPUS_PER_TASK} \
 --mem_mb $((${SLURM_MEM_PER_CPU}*${SLURM_CPUS_PER_TASK}-256)) \
 --fs-license-file ${FS_LICENSE} \
 --fs-subjects-dir /home/javan/scratch/prep_all_part2/derivatives/sub-MTL0003/ses-FU48A/anat \
 --bids-filter-file /jbid/config_MTL0003_distortion2.json \
 --notrack \
 --skip-bids-validation \
 --output-spaces MNI152NLin2009cAsym:res-2 MNI152NLin2009cAsym anat"

Version:

22.0.0

Environment (Docker, Singularity, custom installation):

Singularity

Data formatted according to a validatable standard? Please provide the output of the validator:

Relevant log outputs (up to 20 lines):

Screenshots / relevant information:

Config file without FMAPS

{"t1w": {
        "datatype": "anat",
        "session": "FU03A",
        "acquisition": null,
        "suffix": "T1w"
    },
    "bold": {
        "datatype": "func",
        "session": "FU03A",
        "task": "rest",
        "suffix": "bold"}}

Config file with FMAPS

{"t1w": {
        "datatype": "anat",
        "session": "FU48A",
        "acquisition": null,
        "suffix": "T1w"
    },
    "bold": {
        "datatype": "func",
        "session": "FU48A",
        "task": "rest",
        "suffix": "bold"},
    "fmap" :  {
        "datatype": "fmap",
        "session": "FU48A"}}

Also to add, I have lunched fMRIPrep on these sets of data without defining the sessions and the inclusion of the config file, the output was fully generated, and the susceptibility distortion correction was applied.

I would highly appreciate it if you could please guide me through navigating the issue.
Regards,
Ali

Steven · March 31, 2023, 2:32pm

Hi @AJavan,

I have relabeled your post as Software Support and reformatted your post accordingly to fit in with the category template. In future software posts, please (1) use this template, and (2) paste the text in your post (using the code button in the text editor </>) instead of including download links.

Some thoughts about this issue:

It is not clear if your data are BIDS valid, what is the output from the BIDS validator?

Were there any errors? Anything relevant in log files?
3)

Are you sure SDC was not applied here? I would imagine if the fmap folder was created then SDC was applied.

If you are still in the testing phase, I would recommend upgrading to version 23.0.1 as they made some changed to how SDC works.

Best,

Steven

AJavan · April 3, 2023, 5:08pm

Hi @Steven,

First, thank you for your reply and the modification on my question I’ll stick to the structure for future.

It is not clear if your data are BIDS valid, what is the output from the BIDS validator?

It is not 100% bidsvalidated but it almost is. There are a few small changes required to be validated by the validator (the structure of the directories and the data there are bids compatibale and it has been fed to fmriprep before with no issue).

Were there any errors? Anything relevant in log files?

The freesurfer directory was not mounted to the singularity which I mounted and re run the pipeline. The anat, fmap, and func directories are created at this point but the SDC is not yet applied.

Are you sure SDC was not applied here? I would imagine if the fmap folder was created then SDC was applied.

Yes, I am sure based on the fMRIPrep html report. Seems like that it reads the fmap data as it demonstrates them after the anatomical section, but as of the appliance of the SDC, in the Functional section it has written : Susceptibility distortion correction: None . Also, no SDC related output has been provided for the functional data in the html report.

You have no suffix specified, maybe that is why there are issues?

I have 3 suffixes (magnitude1, magnitude2, and phasediff) in my fmap source directory structure,so I included them all as follows:

{"t1w": {
	"datatype": "anat",
        "session": "FU03A",
        "acquisition": null,
        "suffix": "T1w"
    },
    "bold": {
	"datatype": "func",
        "session": "FU03A",
        "task": "rest",
        "suffix": "bold"},
    "fmap" :  {
        "datatype": "fmap",
        "session": "FU03A",
        "suffix": "magnitude1",
        "suffix": "magnitude2",
        "suffix": "phasediff"}}

Adding, the suffixes and also mounting the fsdir, fixed the issue of the func directory not being created, but the SDC has not yert been applied on the images.

If you are still in the testing phase, I would recommend upgrading to version 23.0.1 as they made some changed to how SDC works.

I am creating a singularity image with the fMRIPrep version 23.0.1 and I have also tried running the script using a singulairty image of fMRIPrep version 20.2.0 (with and without the config file provided), since I have successfully used it for the same data and as a result, SDC was applied on the images.
With this approach I intend to see whether version 22.0.0 version works differently with SDC which prevents it from being applied on my data.
Running my script using fMRIPrep version 20.2.0 both with and without the config file ( the one mentioned in this message), SDC has been applied on the imge but no fmap folder is created (consistent with my previous run using version 20.2.0). In the Functional section of the fMRIPrep html report has also mentioned Susceptibility distortion correction: FMB (fieldmap-based) - phase-difference map and the SDC report has been provided afterwards in the report.
To add to this, we have another set of data that are multi-echo and with epi as fmap images acquired. I have run fMRIPrep version 22.0.0 on those and SDC was successfully applied on the functional images.

AJavan · April 6, 2023, 3:57pm

Hi @Steven,

I have BIDS validated the dataset (the fmaps were not fully BIDS compatibale, the session were missing in the intendedfor section of the json files in the fmap folder of the dataset, recognized it from this question on neurostars). I added sessions to the json files, the data was BIDS validated further, I inserted suffixes fpr the config file and upgraded the fmriprep singularity image to the vesrion 23.0.1. At this stage, the SDC was successfully applied on the data afeter running fMRIPrep.
Therefore, modifications I did:

Adding session to the intendedfor section of the json files in the fmap folder of the sourcedata and make it BIDS validated.
Added fmap compatibale suffixes to the config file as mentioned in the previous section.
Upgraded the fMRIPrep to vesrion 23.0.1

Many thanks for the tips,
Ali