Fmriprep mritotal failed

Software Support
, ,
1

Summary of what happened:

I am using fmriprep 24.0.0 to preprocess several participants’ MRI data using slurm array, one of the data failed in recond-all step, the error is

spawn: exec of minc_modify_header failed: No such file or directory
	autocrop: crashed while running minc_modify_header (termination status=65280)
	mritotal: crashed while running autocrop (termination status=65280)
	ERROR: mritotal failed, see transforms/talairach.log

Command used (and if a helper script was used, a link to the helper script or the command generated):

#!/bin/bash
#SBATCH --job-name=fmriprep
#SBATCH --time=24:00:00
#SBATCH --cpus-per-task=16
#SBATCH --mem-per-cpu=4G
#SBATCH --mail-type=ALL
#SBATCH --array=0-10 
# ------------------------------------------

module load singularity/3.5.3
module load freesurfer/7.2.0
source $FREESURFER_HOME/SetUpFreeSurfer.sh

STUDY=/work/mindlab/NUBIC/MCI_Study/dcm2bids/BIDS_ben
BIDS_DIR="${STUDY}"
DERIVS_DIR=/derivatives/fmriprep
LOCAL_FREESURFER_DIR="${BIDS_DIR}/derivatives/freesurfer"
CONTAINER_PATH=/shared/container_repository/fmriprep/24.0.0
CONTAINER_NAME=fmriprep_24.0.0.sif

# Prepare some writeable bind-mount points.
TEMPLATEFLOW_HOST_HOME=/scratch/$USER/my_fmriprep/.cache/templateflow
FMRIPREP_HOST_CACHE=/scratch/$USER/my_fmriprep/.cache/fmriprep
mkdir -p ${TEMPLATEFLOW_HOST_HOME}
mkdir -p ${FMRIPREP_HOST_CACHE}

# Prepare derivatives folder
mkdir -p ${BIDS_DIR}/${DERIVS_DIR}
mkdir -p ${LOCAL_FREESURFER_DIR}

echo "Freesurfer has been loaded and set up, the license can be found here $FREESURFER_HOME/license.txt"

# Make sure FS_LICENSE is defined in the container.
export SINGULARITYENV_FS_LICENSE=$FREESURFER_HOME/license.txt

# Designate a templateflow bind-mount point
export SINGULARITYENV_TEMPLATEFLOW_HOME="/templateflow"
SINGULARITY_CMD="singularity run --cleanenv -B /shared:/shared -B $BIDS_DIR:/data -B ${TEMPLATEFLOW_HOST_HOME}:${SINGULARITYENV_TEMPLATEFLOW_HOME} -B /scratch/$USER/my_fmriprep:/work -B ${LOCAL_FREESURFER_DIR}:/fsdir $CONTAINER_PATH/$CONTAINER_NAME"

# Extract all folder names beginning with "sub" as the subjects list
subjects=($(ls -d ${BIDS_DIR}/sub-* | xargs -n 1 basename))

# Access the subject for the current task
subject=${subjects[$SLURM_ARRAY_TASK_ID]}

# Remove IsRunning files from FreeSurfer
find ${LOCAL_FREESURFER_DIR}/${subject}/ -name "*IsRunning*" -type f -delete


# Compose the command line
cmd="${SINGULARITY_CMD} /data /data/${DERIVS_DIR} participant --participant-label ${subject} -w /work/ -vv --omp-nthreads 8 --nthreads 12 --mem_mb 30000 --output-spaces MNI152NLin2009cAsym:res-native --skip-bids-validation --fs-subjects-dir /fsdir"
# Setup done, run the command
echo Running task ${SLURM_ARRAY_TASK_ID}
echo Commandline: $cmd
eval $cmd
exitcode=$?

# Output results to a table
echo "sub-$subject   ${SLURM_ARRAY_TASK_ID}    $exitcode" \
      >> ${SLURM_JOB_NAME}.${SLURM_ARRAY_JOB_ID}tsv
echo Finished tasks ${SLURM_ARRAY_TASK_ID} with exit code $exitcode
exit $exitcode

Version:

fmriprep 24.0.0

Environment (Docker, Singularity / Apptainer, custom installation):

Singularity

Data formatted according to a validatable standard? Please provide the output of the validator:

Singularity> bids-validator --verbose /work/mindlab/NUBIC/MCI_Study/dcm2bids/BIDS_test
bids-validator@1.14.6
        1: [ERR] Files with such naming scheme are not part of BIDS specification. This error is most commonly caused by typos in file names that make them not BIDS compatible. Please consult the specification and make sure your files are named correctly. If this is not a file naming issue (for example when including files not yet covered by the BIDS specification) you should include a ".bidsignore" file in your dataset (see https://github.com/bids-standard/bids-validator#bidsignore for details). Please note that derived (processed) data should be placed in /derivatives folder and source data (such as DICOMS or behavioural logs in proprietary formats) should be placed in the /sourcedata folder. (code: 1 - NOT_INCLUDED)
                ./bids-validator_latest.sif
                        Files with such naming scheme are not part of BIDS specification. This error is most commonly caused by typos in file names that make them not BIDS compatible. Please consult the specification and make sure your files are named correctly. If this is not a file naming issue (for example when including files not yet covered by the BIDS specification) you should include a ".bidsignore" file in your dataset (see https://github.com/bids-standard/bids-validator#bidsignore for details). Please note that derived (processed) data should be placed in /derivatives folder and source data (such as DICOMS or behavioural logs in proprietary formats) should be placed in the /sourcedata folder.
                        Evidence: bids-validator_latest.sif

        Please visit https://neurostars.org/search?q=NOT_INCLUDED for existing conversations about this issue.

        1: [WARN] Task scans should have a corresponding events.tsv file. If this is a resting state scan you can ignore this warning or rename the task to include the word "rest". (code: 25 - EVENTS_TSV_MISSING)
                ./sub-230913Jpri/ses-1/func/sub-230913Jpri_ses-1_task-facename_bold.nii.gz
                        Task scans should have a corresponding events.tsv file. It can be included one of the following locations: /events.tsv, /task-facename_events.tsv, /sub-230913Jpri/sub-230913Jpri_events.tsv, /sub-230913Jpri/sub-230913Jpri_task-facename_events.tsv, /sub-230913Jpri/ses-1/sub-230913Jpri_ses-1_events.tsv, /sub-230913Jpri/ses-1/sub-230913Jpri_ses-1_task-facename_events.tsv, /sub-230913Jpri/ses-1/func/sub-230913Jpri_ses-1_events.tsv, /sub-230913Jpri/ses-1/func/sub-230913Jpri_ses-1_task-facename_events.tsv
                ./sub-230913Jpri/ses-1/func/sub-230913Jpri_ses-1_task-facenametest_bold.nii.gz
                        Task scans should have a corresponding events.tsv file. It can be included one of the following locations: /events.tsv, /task-facenametest_events.tsv, /sub-230913Jpri/sub-230913Jpri_events.tsv, /sub-230913Jpri/sub-230913Jpri_task-facenametest_events.tsv, /sub-230913Jpri/ses-1/sub-230913Jpri_ses-1_events.tsv, /sub-230913Jpri/ses-1/sub-230913Jpri_ses-1_task-facenametest_events.tsv, /sub-230913Jpri/ses-1/func/sub-230913Jpri_ses-1_events.tsv, /sub-230913Jpri/ses-1/func/sub-230913Jpri_ses-1_task-facenametest_events.tsv
                ./sub-230913Jpri/ses-1/func/sub-230913Jpri_ses-1_task-musbid_run-01_bold.nii.gz
                        Task scans should have a corresponding events.tsv file. It can be included one of the following locations: /events.tsv, /task-musbid_events.tsv, /run-01_events.tsv, /task-musbid_run-01_events.tsv, /sub-230913Jpri/sub-230913Jpri_events.tsv, /sub-230913Jpri/sub-230913Jpri_task-musbid_events.tsv, /sub-230913Jpri/sub-230913Jpri_run-01_events.tsv, /sub-230913Jpri/sub-230913Jpri_task-musbid_run-01_events.tsv, /sub-230913Jpri/ses-1/sub-230913Jpri_ses-1_events.tsv, /sub-230913Jpri/ses-1/sub-230913Jpri_ses-1_task-musbid_events.tsv, /sub-230913Jpri/ses-1/sub-230913Jpri_ses-1_run-01_events.tsv, /sub-230913Jpri/ses-1/sub-230913Jpri_ses-1_task-musbid_run-01_events.tsv, /sub-230913Jpri/ses-1/func/sub-230913Jpri_ses-1_events.tsv, /sub-230913Jpri/ses-1/func/sub-230913Jpri_ses-1_task-musbid_events.tsv, /sub-230913Jpri/ses-1/func/sub-230913Jpri_ses-1_run-01_events.tsv, /sub-230913Jpri/ses-1/func/sub-230913Jpri_ses-1_task-musbid_run-01_events.tsv
                ./sub-230913Jpri/ses-1/func/sub-230913Jpri_ses-1_task-musbid_run-02_bold.nii.gz
                        Task scans should have a corresponding events.tsv file. It can be included one of the following locations: /events.tsv, /task-musbid_events.tsv, /run-02_events.tsv, /task-musbid_run-02_events.tsv, /sub-230913Jpri/sub-230913Jpri_events.tsv, /sub-230913Jpri/sub-230913Jpri_task-musbid_events.tsv, /sub-230913Jpri/sub-230913Jpri_run-02_events.tsv, /sub-230913Jpri/sub-230913Jpri_task-musbid_run-02_events.tsv, /sub-230913Jpri/ses-1/sub-230913Jpri_ses-1_events.tsv, /sub-230913Jpri/ses-1/sub-230913Jpri_ses-1_task-musbid_events.tsv, /sub-230913Jpri/ses-1/sub-230913Jpri_ses-1_run-02_events.tsv, /sub-230913Jpri/ses-1/sub-230913Jpri_ses-1_task-musbid_run-02_events.tsv, /sub-230913Jpri/ses-1/func/sub-230913Jpri_ses-1_events.tsv, /sub-230913Jpri/ses-1/func/sub-230913Jpri_ses-1_task-musbid_events.tsv, /sub-230913Jpri/ses-1/func/sub-230913Jpri_ses-1_run-02_events.tsv, /sub-230913Jpri/ses-1/func/sub-230913Jpri_ses-1_task-musbid_run-02_events.tsv
                ./sub-230913Jpri/ses-1/func/sub-230913Jpri_ses-1_task-musbid_run-03_bold.nii.gz
                        Task scans should have a corresponding events.tsv file. It can be included one of the following locations: /events.tsv, /task-musbid_events.tsv, /run-03_events.tsv, /task-musbid_run-03_events.tsv, /sub-230913Jpri/sub-230913Jpri_events.tsv, /sub-230913Jpri/sub-230913Jpri_task-musbid_events.tsv, /sub-230913Jpri/sub-230913Jpri_run-03_events.tsv, /sub-230913Jpri/sub-230913Jpri_task-musbid_run-03_events.tsv, /sub-230913Jpri/ses-1/sub-230913Jpri_ses-1_events.tsv, /sub-230913Jpri/ses-1/sub-230913Jpri_ses-1_task-musbid_events.tsv, /sub-230913Jpri/ses-1/sub-230913Jpri_ses-1_run-03_events.tsv, /sub-230913Jpri/ses-1/sub-230913Jpri_ses-1_task-musbid_run-03_events.tsv, /sub-230913Jpri/ses-1/func/sub-230913Jpri_ses-1_events.tsv, /sub-230913Jpri/ses-1/func/sub-230913Jpri_ses-1_task-musbid_events.tsv, /sub-230913Jpri/ses-1/func/sub-230913Jpri_ses-1_run-03_events.tsv, /sub-230913Jpri/ses-1/func/sub-230913Jpri_ses-1_task-musbid_run-03_events.tsv
                ./sub-230913Jpri/ses-3/func/sub-230913Jpri_ses-3_task-facename_bold.nii.gz
                        Task scans should have a corresponding events.tsv file. It can be included one of the following locations: /events.tsv, /task-facename_events.tsv, /sub-230913Jpri/sub-230913Jpri_events.tsv, /sub-230913Jpri/sub-230913Jpri_task-facename_events.tsv, /sub-230913Jpri/ses-3/sub-230913Jpri_ses-3_events.tsv, /sub-230913Jpri/ses-3/sub-230913Jpri_ses-3_task-facename_events.tsv, /sub-230913Jpri/ses-3/func/sub-230913Jpri_ses-3_events.tsv, /sub-230913Jpri/ses-3/func/sub-230913Jpri_ses-3_task-facename_events.tsv
                ./sub-230913Jpri/ses-3/func/sub-230913Jpri_ses-3_task-facenametest_bold.nii.gz
                        Task scans should have a corresponding events.tsv file. It can be included one of the following locations: /events.tsv, /task-facenametest_events.tsv, /sub-230913Jpri/sub-230913Jpri_events.tsv, /sub-230913Jpri/sub-230913Jpri_task-facenametest_events.tsv, /sub-230913Jpri/ses-3/sub-230913Jpri_ses-3_events.tsv, /sub-230913Jpri/ses-3/sub-230913Jpri_ses-3_task-facenametest_events.tsv, /sub-230913Jpri/ses-3/func/sub-230913Jpri_ses-3_events.tsv, /sub-230913Jpri/ses-3/func/sub-230913Jpri_ses-3_task-facenametest_events.tsv
                ./sub-230913Jpri/ses-3/func/sub-230913Jpri_ses-3_task-musbid_bold.nii.gz
                        Task scans should have a corresponding events.tsv file. It can be included one of the following locations: /events.tsv, /task-musbid_events.tsv, /sub-230913Jpri/sub-230913Jpri_events.tsv, /sub-230913Jpri/sub-230913Jpri_task-musbid_events.tsv, /sub-230913Jpri/ses-3/sub-230913Jpri_ses-3_events.tsv, /sub-230913Jpri/ses-3/sub-230913Jpri_ses-3_task-musbid_events.tsv, /sub-230913Jpri/ses-3/func/sub-230913Jpri_ses-3_events.tsv, /sub-230913Jpri/ses-3/func/sub-230913Jpri_ses-3_task-musbid_events.tsv

        Please visit https://neurostars.org/search?q=EVENTS_TSV_MISSING for existing conversations about this issue.

        2: [WARN] Not all subjects/sessions/runs have the same scanning parameters. (code: 39 - INCONSISTENT_PARAMETERS)
                ./sub-230913Jpri/ses-1/func/sub-230913Jpri_ses-1_task-musbid_run-02_bold.nii.gz
                         The most common set of dimensions is: 80,80,48,121 (voxels), This file has the dimensions: 80,80,48,45 (voxels).
                ./sub-230913Jpri/ses-1/func/sub-230913Jpri_ses-1_task-musbid_run-03_bold.nii.gz
                         The most common set of dimensions is: 80,80,48,121 (voxels), This file has the dimensions: 80,80,48,1440 (voxels).

        Please visit https://neurostars.org/search?q=INCONSISTENT_PARAMETERS for existing conversations about this issue.

        3: [WARN] The Authors field of dataset_description.json should contain an array of fields - with one author per field. This was triggered based on the presence of only one author field. Please ign
ore if all contributors are already properly listed. (code: 102 - TOO_FEW_AUTHORS)

        Please visit https://neurostars.org/search?q=TOO_FEW_AUTHORS for existing conversations about this issue.

        4: [WARN] The Name field of dataset_description.json is present but empty of visible characters. (code: 115 - EMPTY_DATASET_NAME)

        Please visit https://neurostars.org/search?q=EMPTY_DATASET_NAME for existing conversations about this issue.


        Summary:                Available Tasks:        Available Modalities: 
        62 Files, 3.41GB        facename                MRI                   
        1 - Subject             facenametest                                  
        2 - Sessions            musbid                                        
                                rest

Relevant log outputs (up to 20 lines):

writing to transforms/tmp.talairach.4992/src.mnc...
	--------------------------------------------
	mritotal -verbose -debug -clobber -modeldir /opt/freesurfer/mni/bin/../share/mni_autoreg -protocol icbm transforms/tmp.talairach.4992/src.mnc transforms/talairach.auto.xfm
	Transforming slices:.................................................................................................................................................................Done
	spawn: exec of minc_modify_header failed: No such file or directory
	autocrop: crashed while running minc_modify_header (termination status=65280)
	mritotal: crashed while running autocrop (termination status=65280)
	ERROR: mritotal failed, see transforms/talairach.log
	Linux c0324 3.10.0-1160.25.1.el7.x86_64 #1 SMP Wed Apr 28 21:49:45 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

	recon-all -s sub-230913Jpri exited with ERRORS at Sat Jun 29 17:14:59 EDT 2024

	For more details, see the log file /fsdir/sub-230913Jpri/scripts/recon-all.log
	To report a problem, see http://surfer.nmr.mgh.harvard.edu/fswiki/BugReporting

Stderr:

Traceback:
	RuntimeError: subprocess exited with code 1.

Screenshots / relevant information:

2

Hi @Jinyu_WANG, and welcome to neurostars!

Looks like a freesurfer command minc_modify_header was excluded from the fmriprep container. However, mri_total should only be running on images that were poor quality and thus had to go through some extra recon-all steps. You could run freesurfer externally and then pass that into fmriprep with the —fs-subjects-dir argument, but I would see first if that subject’s data is even usable.

Best,
Steven

3

Hi Steven,

Thank you for your response, do you know what can be the potential reasons that minc_modify_header was excluded from the fmriprep container? I would appreciate if you could give me suggestions of how to check if this subjects’ data is usable! Many thanks!

Best,
Jinyu

4

Hi @Jinyu_WANG,

The function was probably excluded to reduce the filesize of the container, and in testing it was never used because image quality was good enough that it was not needed.

You can visualize the T1w in an image viewer, or run MRIQC on it (https://mriqc.readthedocs.io/en/latest/. The Euler number is usually a good indicator of quality, but there are other image quality metrics output as well.

Best,
Steven

1 Like
5

If you really want to, and you know how, you can make a fork of fMRIPrep, remove freesurfer/mni/bin/minc_modify_header from fmriprep/docker/files/freesurfer7.3.2-exclude.txt at ff1b199c8a0ef1e019c9b9bf4e6a11d2eda42773 · nipreps/fmriprep · GitHub, and rebuild the docker container from your branch, but it’s possible there are other functions that could be missing in this alternate workflow.

1 Like
6

Compared to other participants who run successfully, this is an old adult and the ventricle of this participant is very large, would this cause the Talariach Failure during freeesurfer recon-all? The recon-all.log for this participant showed the TAL QA check failed, it recommends manual TAL alignment.

7

It could be, but I am not very familiar with the failure modes of FreeSurfer.

1 Like