FMRIPREP - working_dir issues - changed during preproc

1

Hi all!

I´m currently preprocessing my data with fmriPREP 20.0.2 (singularity) - after some server issues (memory) I had to move my working_dir (-w) from dir xyz/ to another location, i.e., abc/

When trying to proceed after the crash and after having set my -w to abc/ it seems that frmiPrep has some troubles finding files in my working_dir, i.e., messages suggest that some files/workflows/reports point to my old dir xyz/ - which does not exist any longer.
I´m attaching my latest error message when trying to re-run one subj.

Node: fmriprep_wf.single_subject_aws482_wf.anat_preproc_wf.anat_derivatives_wf.t1w_name
Working directory: /data/hippocampus/tmp/Flex_Dominik/tempdir_fmriprep/fmriprep_wf/single_subject_aws482_wf/anat_preproc_wf/anat_derivatives_wf/t1w_name
Node inputs: function_str = def fix_multi_T1w_source_name(in_files):
“”"
Make up a generic source name when there are multiple T1s

>>> fix_multi_T1w_source_name([
...     '/path/to/sub-045_ses-test_T1w.nii.gz',
...     '/path/to/sub-045_ses-retest_T1w.nii.gz'])
'/path/to/sub-045_T1w.nii.gz'

"""
import os
from nipype.utils.filemanip import filename_to_list
base, in_file = os.path.split(filename_to_list(in_files)[0])
subject_label = in_file.split("_", 1)[0].split("-")[1]
return os.path.join(base, "sub-%s_T1w.nii.gz" % subject_label)

in_files =
Traceback (most recent call last):
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/multiproc.py”, line 67, in run_node
result[“result”] = node.run(updatehash=updatehash)
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py”, line 443, in run
cached, updated = self.is_cached()
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py”, line 332, in is_cached
hashed_inputs, hashvalue = self._get_hashval()
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py”, line 538, in _get_hashval
self._get_inputs()
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py”, line 580, in _get_inputs
outputs = _load_resultfile(results_fname).outputs
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/utils.py”, line 293, in load_resultfile
result = loadpkl(results_file)
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/utils/filemanip.py”, line 672, in loadpkl
raise e
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/utils/filemanip.py”, line 649, in loadpkl
unpkl = pickle.loads(pkl_contents)
File “/usr/local/miniconda/lib/python3.7/site-packages/traits/has_traits.py”, line 1440, in setstate
self.trait_set( trait_change_notify = trait_change_notify, **state )
File “/usr/local/miniconda/lib/python3.7/site-packages/traits/has_traits.py”, line 1543, in trait_set
setattr( self, name, value )
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/traits_extension.py”, line 329, in validate
value = super(File, self).validate(objekt, name, value, return_pathlike=True)
File “/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/traits_extension.py”, line 134, in validate
self.error(objekt, name, str(value))
File “/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_handlers.py”, line 172, in error
value )
traits.trait_errors.TraitError: The ‘out_report’ trait of a _TemplateDimensionsOutputSpec instance must be a pathlike object or string representing an existing file, but a value of ‘/data/home1/dkraft/tempdir_fmriprep/fmriprep_wf/single_subject_aws482_wf/anat_preproc_wf/anat_template_wf/t1w_ref_dimensions/report.html’ <class ‘str’> was specified.

I´m wondering whether I got something wrong with setting the -w working_dir and this “hard coding” is intended and if so, whether there is an easy way to change any xyz/ leftover to /abc.

Thank you very much in advance!

Dominik

2

Relative paths are not well supported in nipype due to some complications involving symlinks, so we don’t set it as a default, but nipype does support a configuration option execution.use_relative_paths. If you know in advance you’re going to be in a situation you might need to move your working directory, you can create a nipype.cfg file:

[execution]
use_relative_paths = true

This should be placed in ~/.nipype or some directory pointed to by the NIPYPE_CONFIG_DIR environment variable (in case your home directory is not visible in Singularity).

This may also not survive a working directory move, but at least it’s not guaranteed to fail.

3

@effigies: thank you very much for your quick response and clarification! I will give it a try and keep it in mind for upcoming uses!

4

Did it work? Recently did the similar directory move and ran into same issues (on singularity)

5

it didn´t work out for me, even though my home dir should be visible in Singularity :roll_eyes: I just re-run fmriprep in the end

1 Like