Fmriprep 23.0.0 autorecon1 seems to use wrong derivatives path

Software Support
1

Summary of what happened:

Running fmriprep for the first time on institution’s new compute cluster to make sure it is installed correctly. Trying to preprocess one subject that prior to this, was successfully run on fmriprep 20.2.0 at a different institution. When trying to preprocess here, fmriprep errored-out early during autorecon1 saying it is unable to find the correct directory. However, it seemed to be looking in $HOME/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri, not the derivatives directory I specified in the fmriprep arguments.

Command used (and if a helper script was used, a link to the helper script or the command generated):

run via slurm on a cluster, with fmriprep installed as a singularity:

#!/bin/bash

project_dir=/bigdata/lab/sburnslab/MRI_data/DIAD
scratch_dir=/scratch/$SLURM_JOB_USER/$SLURM_JOB_ID

data_dir=$project_dir/data
bids_dir=$data_dir/bids
derivatives_dir=$bids_dir/derivatives
raw_dir=$data_dir/dicom 
defaced_dir=$bids_dir/derivatives/deface 
scripts_dir=$project_dir/code/preprocessing

singularity run --cleanenv --bind $project_dir --bind $scratch_dir \
    /opt/linux/rocky/8/software/fmriprep/23.0.0/bin/fmriprep \
    $bids_dir $derivatives_dir participant \
    --participant-label sub-$1 \
    --fs-license-file $project_dir/code/preprocessing/license.txt \
    --no-submm-recon \
    --bold2t1w-dof 6 \
    --nthreads 8 --omp-nthreads 8 \
    --output-spaces T1w fsaverage:den-41k \
                    MNI152NLin2009cAsym:res-native \
    --write-graph --work-dir $scratch_dir \
    --ignore fieldmaps \
    --use-syn-sdc warn \

Version:

fmriprep 23.0.0

Environment (Docker, Singularity, custom installation):

singularity

Data formatted according to a validatable standard? Please provide the output of the validator:

yes

Relevant log outputs (up to 20 lines):

File: /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sub-022/log/20230329-132126_a7503652-b24a-4e09-a6ef-0a8f0640e557/crash-20230329-133457-smb02011-autorecon1-4ed0aaee-91b8-4263-9535-2a132c1156de.txt
Working Directory: /scratch/smb02011/1218/fmriprep_23_0_wf/single_subject_022_wf/anat_preproc_wf/surface_recon_wf/autorecon1
Inputs:
FLAIR_file:
T1_files:
T2_file:
args:
big_ventricles:
brainstem:
directive: autorecon1
environ: {}
expert:
flags:
hemi:
hippocampal_subfields_T1:
hippocampal_subfields_T2:
hires:
mprage:
mri_aparc2aseg:
mri_ca_label:
mri_ca_normalize:
mri_ca_register:
mri_edit_wm_with_aseg:
mri_em_register:
mri_fill:
mri_mask:
mri_normalize:
mri_pretess:
mri_remove_neck:
mri_segment:
mri_segstats:
mri_tessellate:
mri_watershed:
mris_anatomical_stats:
mris_ca_label:
mris_fix_topology:
mris_inflate:
mris_make_surfaces:
mris_register:
mris_smooth:
mris_sphere:
mris_surf2vol:
mrisp_paint:
openmp: 8
parallel:
steps:
subject_id: recon_all
subjects_dir:
talairach:
use_FLAIR:
use_T2:
xopts:
Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
  File "/opt/conda/lib/python3.9/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node autorecon1.

Cmdline:
	recon-all -autorecon1 -i /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/sub-022/ses-01/anat/sub-022_ses-01_T1w.nii.gz -noskullstrip -noT2pial -noFLAIRpial -openmp 8 -subjid sub-022 -sd /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer
Stdout:
	fs-check-version --s sub-022 --o /tmp/tmp.iUiW3w
	Wed Mar 29 13:31:47 PDT 2023

	setenv SUBJECTS_DIR /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer
	cd /scratch/smb02011/1218/fmriprep_23_0_wf/single_subject_022_wf/anat_preproc_wf/surface_recon_wf/autorecon1
	/opt/freesurfer/bin/fs-check-version --s sub-022 --o /tmp/tmp.iUiW3w
	-rwxrwxr-x. 1 nobody nogroup 18565 Aug  3  2022 /opt/freesurfer/bin/fs-check-version

	freesurfer-linux-ubuntu22_x86_64-7.3.2-20220804-6354275
	$Id$
	Linux a001.hpc.pomona.edu 4.18.0-425.3.1.el8.x86_64 #1 SMP Wed Nov 9 20:13:27 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
	pid 911907
	Current FS Version freesurfer-linux-ubuntu22_x86_64-7.3.2-20220804-6354275
	Subject does not have a bstampfile, copying /opt/freesurfer/build-stamp.txt
	Subject FS Version: freesurfer-linux-ubuntu22_x86_64-7.3.2-20220804-6354275
	No constraints on version because REQ=UnSet and FsVerFile=NotThere
	#@#% fs-check-version match = 1
	fs-check-version Done
	INFO: SUBJECTS_DIR is /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer
	Actual FREESURFER_HOME /opt/freesurfer
	Linux a001.hpc.pomona.edu 4.18.0-425.3.1.el8.x86_64 #1 SMP Wed Nov 9 20:13:27 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
	/bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022

	 mri_convert /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/sub-022/ses-01/anat/sub-022_ses-01_T1w.nii.gz /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri/orig/001.mgz 

	mri_convert /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/sub-022/ses-01/anat/sub-022_ses-01_T1w.nii.gz /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri/orig/001.mgz 
	reading from /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/sub-022/ses-01/anat/sub-022_ses-01_T1w.nii.gz...
	TR=2300.00, TE=0.00, TI=0.00, flip angle=0.00
	i_ras = (0.999563, -0.00715817, 0.0286841)
	j_ras = (0.00336618, 0.991492, 0.130127)
	k_ras = (-0.0293715, -0.129973, 0.991082)
	writing to /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri/orig/001.mgz...
	#--------------------------------------------
	#@# MotionCor Wed Mar 29 13:31:51 PDT 2023
	Found 1 runs
	/bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri/orig/001.mgz
	Checking for (invalid) multi-frame inputs...
	Only one run found so motion
	correction will not be performed. I'll
	copy the run to rawavg and continue.

	 cp /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri/orig/001.mgz /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri/rawavg.mgz 


	 mri_info /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri/rawavg.mgz 

	rawavg.mgz ========================================
	Volume information for /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri/rawavg.mgz
	          type: MGH
	    dimensions: 176 x 256 x 256
	   voxel sizes: 1.000000, 1.000000, 1.000000
	          type: SHORT (4)
	           fov: 256.000
	           dof: 1
	        xstart: -88.0, xend: 88.0
	        ystart: -128.0, yend: 128.0
	        zstart: -128.0, zend: 128.0
	            TR: 2300.00 msec, TE: 0.00 msec, TI: 0.00 msec, flip angle: 0.00 degrees
	       nframes: 1
	       PhEncDir: UNKNOWN
	       FieldStrength: 0.000000
	ras xform present
	    xform info: x_r =   0.9996, y_r =   0.0034, z_r =  -0.0294, c_r =    -0.9821
	              : x_a =  -0.0072, y_a =   0.9915, z_a =  -0.1300, c_a =    -0.0301
	              : x_s =   0.0287, y_s =   0.1301, z_s =   0.9911, c_s =    12.0990

	talairach xfm : 
	Orientation   : RAS
	Primary Slice Direction: axial

	voxel to ras transform:
	                0.9996   0.0034  -0.0294   -85.6149
	               -0.0072   0.9915  -0.1300  -109.6746
	                0.0287   0.1301   0.9911  -133.9400
	                0.0000   0.0000   0.0000     1.0000

	voxel-to-ras determinant 1

	ras to voxel transform:
	                0.9996  -0.0072   0.0287    88.6344
	                0.0034   0.9915   0.1301   126.4588
	               -0.0294  -0.1300   0.9911   115.9762
	                0.0000   0.0000   0.0000     1.0000
	/bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022

	 mri_convert /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri/rawavg.mgz /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri/orig.mgz --conform 

	mri_convert /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri/rawavg.mgz /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri/orig.mgz --conform 
	reading from /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri/rawavg.mgz...
	TR=2300.00, TE=0.00, TI=0.00, flip angle=0.00
	i_ras = (0.999563, -0.00715817, 0.0286841)
	j_ras = (0.00336618, 0.991492, 0.130127)
	k_ras = (-0.0293715, -0.129973, 0.991082)
	changing data type from short to uchar (noscale = 0)...
	MRIchangeType: Building histogram 0 4095 1000, flo=0, fhi=0.999, dest_type=0
	Reslicing using trilinear interpolation 
	writing to /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri/orig.mgz...

	 mri_add_xform_to_header -c /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri/transforms/talairach.xfm /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri/orig.mgz /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri/orig.mgz 

	INFO: extension is mgz

	 mri_info /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri/orig.mgz 

	orig.mgz ========================================
	Volume information for /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri/orig.mgz
	          type: MGH
	    dimensions: 256 x 256 x 256
	   voxel sizes: 1.000000, 1.000000, 1.000000
	          type: UCHAR (0)
	           fov: 256.000
	           dof: 1
	        xstart: -128.0, xend: 128.0
	        ystart: -128.0, yend: 128.0
	        zstart: -128.0, zend: 128.0
	            TR: 2300.00 msec, TE: 0.00 msec, TI: 0.00 msec, flip angle: 0.00 degrees
	       nframes: 1
	       PhEncDir: UNKNOWN
	       FieldStrength: 0.000000
	ras xform present
	    xform info: x_r =  -1.0000, y_r =  -0.0000, z_r =   0.0000, c_r =    -0.9821
	              : x_a =  -0.0000, y_a =  -0.0000, z_a =   1.0000, c_a =    -0.0301
	              : x_s =   0.0000, y_s =  -1.0000, z_s =   0.0000, c_s =    12.0990

	talairach xfm : /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri/transforms/talairach.xfm
	Orientation   : LIA
	Primary Slice Direction: coronal

	voxel to ras transform:
	               -1.0000  -0.0000   0.0000   127.0180
	               -0.0000  -0.0000   1.0000  -128.0301
	                0.0000  -1.0000   0.0000   140.0990
	                0.0000   0.0000   0.0000     1.0000

	voxel-to-ras determinant -1

	ras to voxel transform:
	               -1.0000  -0.0000   0.0000   127.0179
	               -0.0000   0.0000  -1.0000   140.0990
	               -0.0000   1.0000  -0.0000   128.0301
	               -0.0000  -0.0000  -0.0000     1.0000
	#--------------------------------------------
	#@# Talairach Wed Mar 29 13:31:57 PDT 2023
	/bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri

	 mri_nu_correct.mni --no-rescale --i orig.mgz --o orig_nu.mgz --ants-n4 --n 1 --proto-iters 1000 --distance 50 

	/usr/bin/bc
	/bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri
	/opt/freesurfer/bin/mri_nu_correct.mni
	--no-rescale --i orig.mgz --o orig_nu.mgz --ants-n4 --n 1 --proto-iters 1000 --distance 50
	nIters 1
	mri_nu_correct.mni 7.3.2
	Linux a001.hpc.pomona.edu 4.18.0-425.3.1.el8.x86_64 #1 SMP Wed Nov 9 20:13:27 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
	Wed Mar 29 13:31:58 PDT 2023
	tmpdir is ./tmp.mri_nu_correct.mni.912473
	cd /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri
	AntsN4BiasFieldCorrectionFs -i orig.mgz -o ./tmp.mri_nu_correct.mni.912473/nu0.mgz --dtype uchar
	AntsN4BiasFieldCorrectionFs done
	mri_convert ./tmp.mri_nu_correct.mni.912473/nu0.mgz orig_nu.mgz --like orig.mgz --conform
	mri_convert ./tmp.mri_nu_correct.mni.912473/nu0.mgz orig_nu.mgz --like orig.mgz --conform 
	reading from ./tmp.mri_nu_correct.mni.912473/nu0.mgz...
	TR=2300.00, TE=0.00, TI=0.00, flip angle=0.00
	i_ras = (-1, -4.65661e-10, 1.86265e-09)
	j_ras = (-1.86265e-09, -1.49012e-08, -1)
	k_ras = (0, 1, 1.49012e-08)
	INFO: transform src into the like-volume: orig.mgz
	writing to orig_nu.mgz...
 
 
	Wed Mar 29 13:34:38 PDT 2023
	mri_nu_correct.mni done

	 talairach_avi --i orig_nu.mgz --xfm transforms/talairach.auto.xfm 

	talairach_avi log file is transforms/talairach_avi.log...
	mv -f /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri/talsrcimg_to_711-2C_as_mni_average_305_t4_vox2vox.txt /bigdata/lab/sburnslab/MRI_data/DIAD/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri/transforms/talsrcimg_to_711-2C_as_mni_average_305_t4_vox2vox.txt
	Started at Wed Mar 29 13:34:39 PDT 2023
	Ended   at Wed Mar 29 13:34:53 PDT 2023
	talairach_avi done

	 cp transforms/talairach.auto.xfm transforms/talairach.xfm 

	lta_convert --src orig.mgz --trg /opt/freesurfer/average/mni305.cor.mgz --inxfm transforms/talairach.xfm --outlta transforms/talairach.xfm.lta --subject fsaverage --ltavox2vox
	7.3.2

	--src: orig.mgz src image (geometry).
	--trg: /opt/freesurfer/average/mni305.cor.mgz trg image (geometry).
	--inmni: transforms/talairach.xfm input MNI/XFM transform.
	--outlta: transforms/talairach.xfm.lta output LTA.
	--s: fsaverage subject name
	--ltavox2vox: output LTA as VOX_TO_VOX transform.
	 LTA read, type : 1
	 1.04093  -0.00647   0.01077   1.99161;
	 0.00518   0.99453   0.25487  -21.67786;
	-0.04030  -0.28154   1.06930  -5.02301;
	 0.00000   0.00000   0.00000   1.00000;
	setting subject to fsaverage
	Writing  LTA to file transforms/talairach.xfm.lta...
	lta_convert successful.
	~/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri/transforms ~/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri 
Stderr:
	/rhome/smb02011/data/bids/derivatives/sourcedata/freesurfer/sub-022/mri: No such file or directory.
Traceback:
	RuntimeError: subprocess exited with code 1.

Screenshots / relevant information:

2

Hi @smburns47 and welcome to neurostars!

Yes, the output layout changed by default in 21.0.0. You can enable the old layout with —output-layout legacy. This will also change where fmriprep looks for Freesurfer data.

Best,
Steven

3

Hi @Steven, thanks for the response!

Is there a way I should do this in 23.0.0 that isn’t legacy? Happy to use the legacy flag, but want to make sure I’m as update-proof as possible since I’m not a particularly advanced fmriprep user.

4

If you have freesurfer outputs you want to use, you can use the --fs-subjects-dir flag to set your own. Keep in mind that fmriprep 20 uses freesurfer version 6, whereas fmriprep 23 uses freesurfer version 7.3.2.

Best,
Steven

5

Isn’t that for pointing to existing freesurfer files to reuse? I’m trying to run a beginning-to-end preprocessing job that includes freesurfer outputs, but I don’t really care what the layout is. I’d just like to configure the fmriprep command so that it executes successfully, wherever it wants to put freesurfer outputs.

6

Ah got it, I thought you already had FS outputs you wanted to reuse. I think the legacy flag is fine for running a whole dataset, and certainly at least worth a shot while you are still testing. Something I am confused by is why fMRIPrep crashed and didn’t simply create the full freesurfer outputs in that sourcedata folder. Maybe it expected the derivatives/sourcedata folder to have already been made?