Fmriprep - combining multi-echo data error - CAMCAN

Hi, I am running CAMCAN (anat+func) data through fmriprep. The functional data is multi-echo and I organised it according to BIDS format etc. fmriprep gave the following error message (below).

I would be grateful for any suggestions/help regarding what has gone wrong!

201116-18:55:42,636 nipype.workflow INFO:
[Node] Setting-up “fmriprep_wf.single_subject_CC120347_wf.func_preproc_task_epimovie_echo_1_wf.bold_bold_trans_wf.bold_reference_wf.validate” in “/work/fmriprep_wf/single_subject_CC120347_wf/func_preproc_task_epimovie_echo_1_wf/bold_bold_trans_wf/bold_reference_wf/bold_file…bids…sub-CC120347…func…sub-CC120347_task-epimovie_echo-3_bold.nii.gz/validate”.
201116-18:55:42,649 nipype.workflow INFO:
[Node] Running “validate” (“niworkflows.interfaces.images.ValidateImage”)
201116-18:55:42,792 nipype.workflow INFO:
[Node] Finished “fmriprep_wf.single_subject_CC120347_wf.func_preproc_task_epimovie_echo_1_wf.bold_bold_trans_wf.bold_reference_wf.validate”.
201116-18:55:46,427 nipype.workflow INFO:
[Node] Setting-up “fmriprep_wf.single_subject_CC120347_wf.func_preproc_task_epimovie_echo_1_wf.bold_bold_trans_wf.bold_reference_wf.gen_ref” in “/work/fmriprep_wf/single_subject_CC120347_wf/func_preproc_task_epimovie_echo_1_wf/bold_bold_trans_wf/bold_reference_wf/bold_file…bids…sub-CC120347…func…sub-CC120347_task-epimovie_echo-3_bold.nii.gz/gen_ref”.
201116-18:55:46,434 nipype.workflow INFO:
[Node] Running “gen_ref” (“niworkflows.interfaces.registration.EstimateReferenceImage”)
201116-18:55:46,645 nipype.workflow INFO:
[Node] Setting-up “fmriprep_wf.single_subject_CC120347_wf.func_preproc_task_epimovie_echo_1_wf.skullstrip_bold_wf.apply_mask” in “/work/fmriprep_wf/single_subject_CC120347_wf/func_preproc_task_epimovie_echo_1_wf/skullstrip_bold_wf/bold_file…bids…sub-CC120347…func…sub-CC120347_task-epimovie_echo-3_bold.nii.gz/apply_mask”.
201116-18:55:46,671 nipype.workflow INFO:
[Node] Running “apply_mask” (“nipype.interfaces.fsl.maths.ApplyMask”), a CommandLine Interface with command:
fslmaths /work/fmriprep_wf/single_subject_CC120347_wf/func_preproc_task_epimovie_echo_1_wf/bold_bold_trans_wf/bold_file…bids…sub-CC120347…func…sub-CC120347_task-epimovie_echo-3_bold.nii.gz/merge/vol0000_xform-00000_merged.nii -mas /work/fmriprep_wf/single_subject_CC120347_wf/func_preproc_task_epimovie_echo_1_wf/skullstrip_bold_wf/bold_file…bids…sub-CC120347…func…sub-CC120347_task-epimovie_echo-3_bold.nii.gz/combine_masks/vol0000_xform-00000_merged_brain_mask_maths.nii.gz /work/fmriprep_wf/single_subject_CC120347_wf/func_preproc_task_epimovie_echo_1_wf/skullstrip_bold_wf/bold_file…bids…sub-CC120347…func…sub-CC120347_task-epimovie_echo-3_bold.nii.gz/apply_mask/vol0000_xform-00000_merged_masked.nii.gz
201116-18:55:46,894 nipype.interface INFO:
stderr 2020-11-16T18:55:46.894259:++ 3dvolreg: AFNI version=Debian-16.2.07~dfsg.1-5~nd16.04+1 (Jun 12 2017) [64-bit]
201116-18:55:46,894 nipype.interface INFO:
stderr 2020-11-16T18:55:46.894259:++ Authored by: RW Cox
201116-18:55:46,900 nipype.interface INFO:
stderr 2020-11-16T18:55:46.900330:** AFNI converts NIFTI_datatype=4 (INT16) in file /work/fmriprep_wf/single_subject_CC120347_wf/func_preproc_task_epimovie_echo_1_wf/bold_bold_trans_wf/bold_reference_wf/bold_file…bids…sub-CC120347…func…sub-CC120347_task-epimovie_echo-3_bold.nii.gz/gen_ref/slice.nii.gz to FLOAT32
201116-18:55:46,900 nipype.interface INFO:
stderr 2020-11-16T18:55:46.900330: Warnings of this type will be muted for this session.
201116-18:55:46,900 nipype.interface INFO:
stderr 2020-11-16T18:55:46.900330: Set AFNI_NIFTI_TYPE_WARN to YES to see them all, NO to see none.
201116-18:55:46,913 nipype.interface INFO:
stderr 2020-11-16T18:55:46.913363:*+ WARNING: If you are performing spatial transformations on an oblique dset,
201116-18:55:46,913 nipype.interface INFO:
stderr 2020-11-16T18:55:46.913363: such as /work/fmriprep_wf/single_subject_CC120347_wf/func_preproc_task_epimovie_echo_1_wf/bold_bold_trans_wf/bold_reference_wf/bold_file…bids…sub-CC120347…func…sub-CC120347_task-epimovie_echo-3_bold.nii.gz/gen_ref/slice.nii.gz,
201116-18:55:46,913 nipype.interface INFO:
stderr 2020-11-16T18:55:46.913363: or viewing/combining it with volumes of differing obliquity,
201116-18:55:46,913 nipype.interface INFO:
stderr 2020-11-16T18:55:46.913363: you should consider running:
201116-18:55:46,913 nipype.interface INFO:
stderr 2020-11-16T18:55:46.913363: 3dWarp -deoblique
201116-18:55:46,913 nipype.interface INFO:
stderr 2020-11-16T18:55:46.913363: on this and other oblique datasets in the same session.
201116-18:55:46,913 nipype.interface INFO:
stderr 2020-11-16T18:55:46.913363: See 3dWarp -help for details.
201116-18:55:46,913 nipype.interface INFO:
stderr 2020-11-16T18:55:46.913578:++ Oblique dataset:/work/fmriprep_wf/single_subject_CC120347_wf/func_preproc_task_epimovie_echo_1_wf/bold_bold_trans_wf/bold_reference_wf/bold_file…bids…sub-CC120347…func…sub-CC120347_task-epimovie_echo-3_bold.nii.gz/gen_ref/slice.nii.gz is 31.599751 degrees from plumb.
201116-18:55:46,914 nipype.interface INFO:
stderr 2020-11-16T18:55:46.914463:++ Coarse del was 10, replaced with 3
201116-18:55:48,602 nipype.workflow INFO:
[Node] Setting-up “fmriprep_wf.single_subject_CC120347_wf.func_preproc_task_epimovie_echo_1_wf.skullstrip_bold_wf.mask_reportlet” in “/work/fmriprep_wf/single_subject_CC120347_wf/func_preproc_task_epimovie_echo_1_wf/skullstrip_bold_wf/bold_file…bids…sub-CC120347…func…sub-CC120347_task-epimovie_echo-3_bold.nii.gz/mask_reportlet”.
201116-18:55:48,611 nipype.workflow INFO:
[Node] Cached “fmriprep_wf.single_subject_CC120347_wf.func_preproc_task_epimovie_echo_1_wf.skullstrip_bold_wf.mask_reportlet” - collecting precomputed outputs
201116-18:55:48,611 nipype.workflow INFO:
[Node] “fmriprep_wf.single_subject_CC120347_wf.func_preproc_task_epimovie_echo_1_wf.skullstrip_bold_wf.mask_reportlet” found cached.
201116-18:55:48,786 nipype.workflow WARNING:
Storing result file without outputs
201116-18:55:48,809 nipype.workflow WARNING:
[Node] Error on “fmriprep_wf.single_subject_CC120347_wf.func_preproc_task_epimovie_echo_1_wf.skullstrip_bold_wf.apply_mask” (/work/fmriprep_wf/single_subject_CC120347_wf/func_preproc_task_epimovie_echo_1_wf/skullstrip_bold_wf/bold_file…bids…sub-CC120347…func…sub-CC120347_task-epimovie_echo-3_bold.nii.gz/apply_mask)
201116-18:55:50,109 nipype.workflow ERROR:
Node apply_mask.a2 failed to run on host nia0733.scinet.local.
201116-18:55:50,116 nipype.workflow ERROR:
Saving crash info to /out/fmriprep/sub-CC120347/log/20201116-185309_573fae44-cfdb-4232-94ef-55c8486b9232/crash-20201116-185550-coughlgi-apply_mask.a2-e2390cb2-bd53-4843-aeb0-6d5f8c408bf9.txt
Traceback (most recent call last):
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/multiproc.py”, line 69, in run_node
result[‘result’] = node.run(updatehash=updatehash)
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py”, line 473, in run
result = self._run_interface(execute=True)
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py”, line 564, in _run_interface
return self._run_command(execute)
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py”, line 649, in _run_command
result = self._interface.run(cwd=outdir)
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py”, line 378, in run
outputs = self.aggregate_outputs(runtime)
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py”, line 455, in aggregate_outputs
predicted_outputs = self._list_outputs() # Predictions from _list_outputs
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/fsl/maths.py”, line 68, in _list_outputs
self.inputs.in_file, suffix=self._suffix)
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/fsl/base.py”, line 242, in _gen_fname
cwd = os.getcwd()
FileNotFoundError: [Errno 2] No such file or directory

201116-18:55:58,516 nipype.workflow INFO:
[Node] Setting-up “fmriprep_wf.single_subject_CC120347_wf.func_preproc_task_epimovie_echo_1_wf.bold_bold_trans_wf.bold_transform” in “/work/fmriprep_wf/single_subject_CC120347_wf/func_preproc_task_epimovie_echo_1_wf/bold_bold_trans_wf/bold_file…bids…sub-CC120347…func…sub-CC120347_task-epimovie_echo-2_bold.nii.gz/bold_transform”.
201116-18:56:00,125 nipype.workflow ERROR:
Node bold_transform.a1 failed to run on host nia1567.scinet.local.
201116-18:56:00,137 nipype.workflow ERROR:
Saving crash info to /out/fmriprep/sub-CC120347/log/20201116-185309_573fae44-cfdb-4232-94ef-55c8486b9232/crash-20201116-185600-coughlgi-bold_transform.a1-1e652d8e-bf5d-4bfc-b660-ad36ef747e63.txt
Traceback (most recent call last):
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/multiproc.py”, line 69, in run_node
result[‘result’] = node.run(updatehash=updatehash)
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py”, line 451, in run
emptydirs(outdir, noexist_ok=True)
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/utils/filemanip.py”, line 866, in emptydirs
raise ex
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/utils/filemanip.py”, line 856, in emptydirs
shutil.rmtree(path)
File “/usr/local/miniconda/lib/python3.7/shutil.py”, line 489, in rmtree
onerror(os.rmdir, path, sys.exc_info())
File “/usr/local/miniconda/lib/python3.7/shutil.py”, line 487, in rmtree
os.rmdir(path)
OSError: [Errno 39] Directory not empty: ‘/work/fmriprep_wf/single_subject_CC120347_wf/func_preproc_task_epimovie_echo_1_wf/bold_bold_trans_wf/bold_file…bids…sub-CC120347…func…sub-CC120347_task-epimovie_echo-2_bold.nii.gz/bold_transform’

201116-18:56:01,807 nipype.workflow INFO:
[Node] Setting-up “fmriprep_wf.single_subject_CC120347_wf.func_preproc_task_epimovie_echo_1_wf.bold_bold_trans_wf.bold_transform” in “/work/fmriprep_wf/single_subject_CC120347_wf/func_preproc_task_epimovie_echo_1_wf/bold_bold_trans_wf/bold_file…bids…sub-CC120347…func…sub-CC120347_task-epimovie_echo-1_bold.nii.gz/bold_transform”.
201116-18:56:04,475 nipype.interface INFO:
stderr 2020-11-16T18:56:04.475082:
201116-18:56:04,475 nipype.interface INFO:
stderr 2020-11-16T18:56:04.475082:Fatal Signal 11 (SIGSEGV) received
201116-18:56:04,475 nipype.interface INFO:
stderr 2020-11-16T18:56:04.475082: 3dvolreg main
201116-18:56:04,475 nipype.interface INFO:
stderr 2020-11-16T18:56:04.475082: Bottom of Debug Stack
201116-18:56:04,475 nipype.interface INFO:
stderr 2020-11-16T18:56:04.475082:** Command line was:
201116-18:56:04,475 nipype.interface INFO:
stderr 2020-11-16T18:56:04.475082:3dvolreg -Fourier -twopass -1Dfile slice.1D -1Dmatrix_save slice.aff12.1D -prefix slice_volreg.nii.gz -zpad 4 -maxdisp1D slice_md.1D /work/fmriprep_wf/single_subject_CC120347_wf/func_preproc_task_epimovie_echo_1_wf/bold_bold_trans_wf/bold_reference_wf/bold_file…bids…sub-CC120347…func…sub-CC120347_task-epimovie_echo-3_bold.nii.gz/gen_ref/slice.nii.gz
201116-18:56:04,475 nipype.interface INFO:
stderr 2020-11-16T18:56:04.475082:** AFNI version = Debian-16.2.07~dfsg.1-5~nd16.04+1 Compile date = Jun 12 2017
201116-18:56:04,475 nipype.interface INFO:
stderr 2020-11-16T18:56:04.475082:** [[Precompiled binary Debian-x86_64-1-gnu: Jun 12 2017]]
201116-18:56:04,475 nipype.interface INFO:
stderr 2020-11-16T18:56:04.475082:** Program Death **
201116-18:56:04,475 nipype.interface INFO:
stderr 2020-11-16T18:56:04.475082:** If you report this crash to the AFNI message board,
201116-18:56:04,475 nipype.interface INFO:
stderr 2020-11-16T18:56:04.475082:** please copy the error messages EXACTLY, and give
201116-18:56:04,475 nipype.interface INFO:
stderr 2020-11-16T18:56:04.475082:** the command line you used to run the program, and
201116-18:56:04,475 nipype.interface INFO:
stderr 2020-11-16T18:56:04.475082:** any other information needed to repeat the problem.
201116-18:56:04,475 nipype.interface INFO:
stderr 2020-11-16T18:56:04.475082:** You may later be asked to upload data to help debug.
201116-18:56:04,481 nipype.interface INFO:
stderr 2020-11-16T18:56:04.481318:** Crash log is appended to file /scratch/c/cgrady/coughlgi/tmp/.afni.crashlog
201116-18:56:04,572 nipype.workflow WARNING:
Storing result file without outputs
201116-18:56:04,577 nipype.workflow WARNING:
[Node] Error on “fmriprep_wf.single_subject_CC120347_wf.func_preproc_task_epimovie_echo_1_wf.bold_bold_trans_wf.bold_reference_wf.gen_ref” (/work/fmriprep_wf/single_subject_CC120347_wf/func_preproc_task_epimovie_echo_1_wf/bold_bold_trans_wf/bold_reference_wf/bold_file…bids…sub-CC120347…func…sub-CC120347_task-epimovie_echo-3_bold.nii.gz/gen_ref)
201116-18:56:06,122 nipype.workflow ERROR:
Node gen_ref.a2 failed to run on host nia0733.scinet.local.
201116-18:56:06,122 nipype.workflow ERROR:
Saving crash info to /out/fmriprep/sub-CC120347/log/20201116-185309_573fae44-cfdb-4232-94ef-55c8486b9232/crash-20201116-185606-coughlgi-gen_ref.a2-d8098684-ad7d-4d04-94ea-1cafd70d0234.txt
Traceback (most recent call last):
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/multiproc.py”, line 69, in run_node
result[‘result’] = node.run(updatehash=updatehash)
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py”, line 473, in run
result = self._run_interface(execute=True)
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py”, line 564, in _run_interface
return self._run_command(execute)
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py”, line 649, in _run_command
result = self._interface.run(cwd=outdir)
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py”, line 376, in run
runtime = self._run_interface(runtime)
File “/usr/local/miniconda/lib/python3.7/site-packages/niworkflows/interfaces/registration.py”, line 411, in _run_interface
zpad=4, outputtype=‘NIFTI_GZ’).run()
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py”, line 376, in run
runtime = self._run_interface(runtime)
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/afni/base.py”, line 115, in _run_interface
return super(AFNICommandBase, self)._run_interface(runtime)
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py”, line 761, in _run_interface
self.raise_exception(runtime)
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py”, line 698, in raise_exception
).format(**runtime.dictcopy()))
RuntimeError: Command:
3dvolreg -Fourier -twopass -1Dfile slice.1D -1Dmatrix_save slice.aff12.1D -prefix slice_volreg.nii.gz -zpad 4 -maxdisp1D slice_md.1D /work/fmriprep_wf/single_subject_CC120347_wf/func_preproc_task_epimovie_echo_1_wf/bold_bold_trans_wf/bold_reference_wf/bold_file…bids…sub-CC120347…func…sub-CC120347_task-epimovie_echo-3_bold.nii.gz/gen_ref/slice.nii.gz
Standard output:

Standard error:
++ 3dvolreg: AFNI version=Debian-16.2.07~dfsg.1-5~nd16.04+1 (Jun 12 2017) [64-bit]
++ Authored by: RW Cox
** AFNI converts NIFTI_datatype=4 (INT16) in file /work/fmriprep_wf/single_subject_CC120347_wf/func_preproc_task_epimovie_echo_1_wf/bold_bold_trans_wf/bold_reference_wf/bold_file…bids…sub-CC120347…func…sub-CC120347_task-epimovie_echo-3_bold.nii.gz/gen_ref/slice.nii.gz to FLOAT32
Warnings of this type will be muted for this session.
Set AFNI_NIFTI_TYPE_WARN to YES to see them all, NO to see none.
*+ WARNING: If you are performing spatial transformations on an oblique dset,
such as /work/fmriprep_wf/single_subject_CC120347_wf/func_preproc_task_epimovie_echo_1_wf/bold_bold_trans_wf/bold_reference_wf/bold_file…bids…sub-CC120347…func…sub-CC120347_task-epimovie_echo-3_bold.nii.gz/gen_ref/slice.nii.gz,
or viewing/combining it with volumes of differing obliquity,
you should consider running:
3dWarp -deoblique
on this and other oblique datasets in the same session.
See 3dWarp -help for details.
++ Oblique dataset:/work/fmriprep_wf/single_subject_CC120347_wf/func_preproc_task_epimovie_echo_1_wf/bold_bold_trans_wf/bold_reference_wf/bold_file…bids…sub-CC120347…func…sub-CC120347_task-epimovie_echo-3_bold.nii.gz/gen_ref/slice.nii.gz is 31.599751 degrees from plumb.
++ Coarse del was 10, replaced with 3

Fatal Signal 11 (SIGSEGV) received
3dvolreg main
Bottom of Debug Stack
** Command line was:
3dvolreg -Fourier -twopass -1Dfile slice.1D -1Dmatrix_save slice.aff12.1D -prefix slice_volreg.nii.gz -zpad 4 -maxdisp1D slice_md.1D /work/fmriprep_wf/single_subject_CC120347_wf/func_preproc_task_epimovie_echo_1_wf/bold_bold_trans_wf/bold_reference_wf/bold_file…bids…sub-CC120347…func…sub-CC120347_task-epimovie_echo-3_bold.nii.gz/gen_ref/slice.nii.gz
** AFNI version = Debian-16.2.07~dfsg.1-5~nd16.04+1 Compile date = Jun 12 2017
** [[Precompiled binary Debian-x86_64-1-gnu: Jun 12 2017]]
** Program Death **
** If you report this crash to the AFNI message board,
** please copy the error messages EXACTLY, and give
** the command line you used to run the program, and
** any other information needed to repeat the problem.
** You may later be asked to upload data to help debug.
** Crash log is appended to file /scratch/c/cgrady/coughlgi/tmp/.afni.crashlog
Return code: 1

201116-18:56:08,124 nipype.workflow ERROR:
Node bold_transform.a0 failed to run on host nia1567.scinet.local.
201116-18:56:08,125 nipype.workflow ERROR:
Saving crash info to /out/fmriprep/sub-CC120347/log/20201116-185309_573fae44-cfdb-4232-94ef-55c8486b9232/crash-20201116-185608-coughlgi-bold_transform.a0-29f81928-6302-4322-9c89-15d33280c89c.txt
Traceback (most recent call last):
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/multiproc.py”, line 69, in run_node
result[‘result’] = node.run(updatehash=updatehash)
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py”, line 451, in run
emptydirs(outdir, noexist_ok=True)
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/utils/filemanip.py”, line 866, in emptydirs
raise ex
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/utils/filemanip.py”, line 856, in emptydirs
shutil.rmtree(path)
File “/usr/local/miniconda/lib/python3.7/shutil.py”, line 485, in rmtree
_rmtree_safe_fd(fd, path, onerror)
File “/usr/local/miniconda/lib/python3.7/shutil.py”, line 427, in _rmtree_safe_fd
onerror(os.rmdir, fullname, sys.exc_info())
File “/usr/local/miniconda/lib/python3.7/shutil.py”, line 425, in _rmtree_safe_fd
os.rmdir(entry.name, dir_fd=topfd)
OSError: [Errno 39] Directory not empty: ‘tmp-bj46t008’

It seems like this is the first error. It’s a little unclear to me, but it seems like it’s trying to load a cached node/subworkflow from a previous run. Is there any chance this is the result of re-running fMRIPrep on the same data without clearing out the working directory?

Hi,
Thanks for your reply! I deleted an old workdir before starting the job so that shouldn’t be the problem!

MANAGED BY INCF