Problem with antsRegistration: FileNotFoundError: No such file or directory fmap_syn0Warp.nii.gz for output 'forward_transforms' of a FixHeaderRegistration interface

Trying to run fmriprep 23.1.4 on one subject on one of our big compute nodes.

  • fMRIPrep version: 23.1.4
  • fMRIPrep command: /opt/conda/envs/fmriprep/bin/fmriprep /dartfs-hpc/rc/lab/C/CANlab/labdata/data/WASABI/1080_wasabi /dartfs-hpc/rc/lab/C/CANlab/labdata/data/WASABI/derivatives/fmriprep participant --participant_label SID001651 --ignore slicetiming --resource-monitor --bold2t1w-dof 9 --dummy-scans 6 --notrack --nprocs 100 --omp-nthreads 8 --nthreads 100 --mem_mb 480000 --fs-license-file /optnfs/freesurfer/6.0.0/license.txt --cifti-output 91k --output-spaces T1w MNI152NLin2009cAsym fsnative fsaverage6 --skull-strip-fixed-seed -w /scratch/f003z4j/fmriprep-work/work-SID001651 --random-seed 42 --stop-on-first-crash --force-syn --use-syn-sdc --skip_bids_validation

Cmdline:
antsRegistration --collapse-output-transforms 1 --dimensionality 3 --initialize-transforms-per-stage 0 --interpolation Linear --output fmap_syn --transform SyN[ 0.8, 6.0, 3.0 ] --metric Mattes[ /scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/syn_preprocessing_auto_00108/anat2epi/clipped_noise_corrected_trans.nii.gz, /scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/fmap_preproc_wf/wf_auto_00108/clip_epi/clipped.nii.gz, 0.5, 48, Random, 0.8 ] --metric Mattes[ /scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/fmap_preproc_wf/wf_auto_00108/lap_anat_norm/clipped_noise_corrected_trans_maths_norm.nii.gz, /scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/fmap_preproc_wf/wf_auto_00108/lap_epi_norm/clipped_maths_norm.nii.gz, 0.5, 48, Random, 0.8 ] --convergence [ 200x100, 1e-06, 5 ] --smoothing-sigmas 2.0x0.0vox --shrink-factors 1x1 --use-histogram-matching 1 --restrict-deformation 0.1x1.0x0.1 --masks [ /scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/syn_preprocessing_auto_00108/mask2epi/sub-SID001651_ses-08_acq-MPRAGEXp3X08mm_run-01_T1w_noise_corrected_ras_template_corrected_xform_rbrainmask_trans_uint8.nii.gz, /scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/fmap_preproc_wf/wf_auto_00108/epi_umask/clipped_mask_union.nii.gz ] --transform SyN[ 0.8, 2.0, 1.0 ] --metric Mattes[ /scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/syn_preprocessing_auto_00108/anat2epi/clipped_noise_corrected_trans.nii.gz, /scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/fmap_preproc_wf/wf_auto_00108/clip_epi/clipped.nii.gz, 0.5, 48 ] --metric Mattes[ /scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/fmap_preproc_wf/wf_auto_00108/lap_anat_norm/clipped_noise_corrected_trans_maths_norm.nii.gz, /scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/fmap_preproc_wf/wf_auto_00108/lap_epi_norm/clipped_maths_norm.nii.gz, 0.5, 48 ] --convergence [ 10, 1e-08, 2 ] --smoothing-sigmas 0.0vox --shrink-factors 1 --use-histogram-matching 1 --restrict-deformation 0.1x1.0x0.1 --masks [ /scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/syn_preprocessing_auto_00108/mask2epi/sub-SID001651_ses-08_acq-MPRAGEXp3X08mm_run-01_T1w_noise_corrected_ras_template_corrected_xform_rbrainmask_trans_uint8.nii.gz, /scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/fmap_preproc_wf/wf_auto_00108/epi_umask/clipped_mask_union.nii.gz ] --winsorize-image-intensities [ 0.001, 0.998 ] --write-composite-transform 0

Errors

Node Name: fmriprep_23_1_wf.single_subject_SID001651_wf.fmap_preproc_wf.wf_auto_00064.syn

File: /dartfs-hpc/rc/lab/C/CANlab/labdata/data/WASABI/derivatives/fmriprep/sub-SID001651/log/20230925-113307_abf325e2-c3fd-4436-9618-199458b214c0/crash-20230926-074131-f003z4j-syn-7ce04c62-8438-4475-b5b7-bccb9e8e257a.txt
Working Directory: /scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/fmap_preproc_wf/wf_auto_00064/syn
Inputs:

  • args:``
  • collapse_output_transforms: True
  • convergence_threshold: [1e-06, 1e-08]
  • convergence_window_size: [5, 2]
  • dimension: 3
  • environ: {'NSLOTS': '8'}
  • fixed_image:``
  • fixed_image_mask:``
  • fixed_image_masks:``
  • float:``
  • initial_moving_transform:``
  • initial_moving_transform_com:``
  • initialize_transforms_per_stage: False
  • interpolation: Linear
  • interpolation_parameters:``
  • invert_initial_moving_transform:``
  • metric: [['Mattes', 'Mattes'], ['Mattes', 'Mattes']]
  • metric_item_trait:``
  • metric_stage_trait:``
  • metric_weight: [[0.5, 0.5], [0.5, 0.5]]
  • metric_weight_item_trait: 1.0
  • metric_weight_stage_trait:``
  • moving_image:``
  • moving_image_mask:``
  • moving_image_masks:``
  • num_threads: 8
  • number_of_iterations: [[200, 100], [10]]
  • output_inverse_warped_image: False
  • output_transform_prefix: fmap_syn
  • output_warped_image: False
  • radius_bins_item_trait: 5
  • radius_bins_stage_trait:``
  • radius_or_number_of_bins: [[48, 48], [48, 48]]
  • random_seed:``
  • restore_state:``
  • restrict_deformation:``
  • sampling_percentage: [[0.8, 0.8], [1.0, 1.0]]
  • sampling_percentage_item_trait:``
  • sampling_percentage_stage_trait:``
  • sampling_strategy: [['Random', 'Random'], [None, None]]
  • sampling_strategy_item_trait:``
  • sampling_strategy_stage_trait:``
  • save_state:``
  • shrink_factors: [[1, 1], [1]]
  • sigma_units: ['vox', 'vox']
  • smoothing_sigmas: [[2.0, 0.0], [0.0]]
  • transform_parameters: [(0.8, 6.0, 3.0), (0.8, 2.0, 1.0)]
  • transforms: ['SyN', 'SyN']
  • use_estimate_learning_rate_once:``
  • use_histogram_matching: [True, True]
  • verbose: False
  • winsorize_lower_quantile: 0.001
  • winsorize_upper_quantile: 0.998
  • write_composite_transform: False

Traceback (most recent call last): File “/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py”, line 67, in run_node result[“result”] = node.run(updatehash=updatehash) File “/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py”, line 527, in run result = self._run_interface(execute=True) File “/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py”, line 645, in _run_interface return self._run_command(execute) File “/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py”, line 771, in _run_command raise NodeExecutionError(msg) nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node syn. Cmdline: antsRegistration --collapse-output-transforms 1 --dimensionality 3 --initialize-transforms-per-stage 0 --interpolation Linear --output fmap_syn --transform SyN[ 0.8, 6.0, 3.0 ] --metric Mattes[ /scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/syn_preprocessing_auto_00064/anat2epi/clipped_noise_corrected_trans.nii.gz, /scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/fmap_preproc_wf/wf_auto_00064/clip_epi/clipped.nii.gz, 0.5, 48, Random, 0.8 ] --metric Mattes[ /scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/fmap_preproc_wf/wf_auto_00064/lap_anat_norm/clipped_noise_corrected_trans_maths_norm.nii.gz, /scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/fmap_preproc_wf/wf_auto_00064/lap_epi_norm/clipped_maths_norm.nii.gz, 0.5, 48, Random, 0.8 ] --convergence [ 200x100, 1e-06, 5 ] --smoothing-sigmas 2.0x0.0vox --shrink-factors 1x1 --use-histogram-matching 1 --restrict-deformation 0.1x1.0x0.1 --masks [ /scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/syn_preprocessing_auto_00064/mask2epi/sub-SID001651_ses-08_acq-MPRAGEXp3X08mm_run-01_T1w_noise_corrected_ras_template_corrected_xform_rbrainmask_trans_uint8.nii.gz, /scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/fmap_preproc_wf/wf_auto_00064/epi_umask/clipped_mask_union.nii.gz ] --transform SyN[ 0.8, 2.0, 1.0 ] --metric Mattes[ /scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/syn_preprocessing_auto_00064/anat2epi/clipped_noise_corrected_trans.nii.gz, /scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/fmap_preproc_wf/wf_auto_00064/clip_epi/clipped.nii.gz, 0.5, 48 ] --metric Mattes[ /scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/fmap_preproc_wf/wf_auto_00064/lap_anat_norm/clipped_noise_corrected_trans_maths_norm.nii.gz, /scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/fmap_preproc_wf/wf_auto_00064/lap_epi_norm/clipped_maths_norm.nii.gz, 0.5, 48 ] --convergence [ 10, 1e-08, 2 ] --smoothing-sigmas 0.0vox --shrink-factors 1 --use-histogram-matching 1 --restrict-deformation 0.1x1.0x0.1 --masks [ /scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/syn_preprocessing_auto_00064/mask2epi/sub-SID001651_ses-08_acq-MPRAGEXp3X08mm_run-01_T1w_noise_corrected_ras_template_corrected_xform_rbrainmask_trans_uint8.nii.gz, /scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/fmap_preproc_wf/wf_auto_00064/epi_umask/clipped_mask_union.nii.gz ] --winsorize-image-intensities [ 0.001, 0.998 ] --write-composite-transform 0 Stdout: Stderr: Traceback: Traceback (most recent call last): File “/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py”, line 453, in aggregate_outputs setattr(outputs, key, val) File “/opt/conda/envs/fmriprep/lib/python3.10/site-packages/traits/trait_types.py”, line 2699, in validate return TraitListObject(self, object, name, value) File “/opt/conda/envs/fmriprep/lib/python3.10/site-packages/traits/trait_list_object.py”, line 582, in init super().init( File “/opt/conda/envs/fmriprep/lib/python3.10/site-packages/traits/trait_list_object.py”, line 213, in init super().init(self.item_validator(item) for item in iterable) File “/opt/conda/envs/fmriprep/lib/python3.10/site-packages/traits/trait_list_object.py”, line 213, in super().init(self.item_validator(item) for item in iterable) File “/opt/conda/envs/fmriprep/lib/python3.10/site-packages/traits/trait_list_object.py”, line 865, in _item_validator return trait_validator(object, self.name, value) File “/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/interfaces/base/traits_extension.py”, line 330, in validate value = super(File, self).validate(objekt, name, value, return_pathlike=True) File “/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/interfaces/base/traits_extension.py”, line 135, in validate self.error(objekt, name, str(value)) File “/opt/conda/envs/fmriprep/lib/python3.10/site-packages/traits/base_trait_handler.py”, line 74, in error raise TraitError( traits.trait_errors.TraitError: Each element of the ‘forward_transforms’ trait of a RegistrationOutputSpec instance must be a pathlike object or string representing an existing file, but a value of ‘/scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/fmap_preproc_wf/wf_auto_00064/syn/fmap_syn0Warp.nii.gz’ was specified. During handling of the above exception, another exception occurred: Traceback (most recent call last): File “/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py”, line 400, in run outputs = self.aggregate_outputs(runtime) File “/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py”, line 460, in aggregate_outputs raise FileNotFoundError(msg) FileNotFoundError: No such file or directory ‘[’/scratch/f003z4j/fmriprep-work/work-SID001651/fmriprep_23_1_wf/single_subject_SID001651_wf/fmap_preproc_wf/wf_auto_00064/syn/fmap_syn0Warp.nii.gz’]’ for output ‘forward_transforms’ of a FixHeaderRegistration interface