Fmriprep Error when performing AROMA and select --template-resampling-grid 2mm

1

Dear Experts

I have tried to run fmriprep with AROMA.
The processing was successful if I did not specifiy --template-resampling-grid 2mm.

However, I tried to use AROMA and 2mm reference grid once with version 1.0.9 and once with 1.0.11, and they both failed.

Is there any advice on how I can solve this problem and successfully perform AROMA with 2mm grid?

Thank you very much!

Chen-Chia

The below is the Error from the terminal:

180420-06:57:53,658 workflow WARNING:
	 [Node] Error on "fmriprep_wf.single_subject_001_wf.func_preproc_ses_postdose_task_rest_wf.ica_aroma_wf.ica_aroma" (/root/src/fmriprep/work/fmriprep_wf/single_subject_001_wf/func_preproc_ses_postdose_task_rest_wf/ica_aroma_wf/ica_aroma)
Traceback (most recent call last):
  File "/usr/local/miniconda/bin/fmriprep", line 11, in <module>
    load_entry_point('fmriprep==1.0.11', 'console_scripts', 'fmriprep')()
  File "/usr/local/miniconda/lib/python3.6/site-packages/fmriprep/cli/run.py", line 274, in main
    fmriprep_wf.run(**plugin_settings)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/workflows.py", line 602, in run
    runner.run(execgraph, updatehash=updatehash, config=self.config)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/plugins/base.py", line 168, in run
    self._clean_queue(jobid, graph, result=result))
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/plugins/base.py", line 227, in _clean_queue
    raise RuntimeError("".join(result['traceback']))
RuntimeError: Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/plugins/multiproc.py", line 68, in run_node
    result['result'] = node.run(updatehash=updatehash)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/nodes.py", line 487, in run
    result = self._run_interface(execute=True)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/nodes.py", line 571, in _run_interface
    return self._run_command(execute)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/nodes.py", line 650, in _run_command
    result = self._interface.run(cwd=outdir)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/interfaces/base/core.py", line 516, in run
    runtime = self._run_interface(runtime)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/interfaces/report_base.py", line 51, in _run_interface
    ReportCapableInterface, self)._run_interface(runtime)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/interfaces/base/core.py", line 1023, in _run_interface
    self.raise_exception(runtime)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/interfaces/base/core.py", line 960, in raise_exception
    ).format(**runtime.dictcopy()))
RuntimeError: Command:
ICA_AROMA.py -den nonaggr -i /root/src/fmriprep/work/fmriprep_wf/single_subject_001_wf/func_preproc_ses_postdose_task_rest_wf/ica_aroma_wf/smooth/vol0000_xform-00000_merged_smooth.nii.gz -meldir /root/src/fmriprep/work/fmriprep_wf/single_subject_001_wf/func_preproc_ses_postdose_task_rest_wf/ica_aroma_wf/melodic -mc /root/src/fmriprep/work/fmriprep_wf/single_subject_001_wf/func_preproc_ses_postdose_task_rest_wf/bold_hmc_wf/normalize_motion/motion_params.txt -o /root/src/fmriprep/work/fmriprep_wf/single_subject_001_wf/func_preproc_ses_postdose_task_rest_wf/ica_aroma_wf/ica_aroma
Standard output:

------------------------------- RUNNING ICA-AROMA ------------------------------- 
--------------- 'ICA-based Automatic Removal Of Motion Artifacts' --------------- 

Step 1) MELODIC
  - The existing/specified MELODIC directory will be used.
  - The MELODIC directory does not contain the required 'stats' folder. Mixture modeling on the Z-statistical maps will be run.
Step 2) Automatic classification of the components
  - registering the spatial maps to MNI
  - extracting the CSF & Edge fraction features
Standard error:
Traceback (most recent call last):
  File "/opt/ICA-AROMA/ICA_AROMA.py", line 200, in <module>
    edgeFract, csfFract = aromafunc.feature_spatial(fslDir, outDir, scriptDir, melIC_MNI)
  File "/opt/ICA-AROMA/ICA_AROMA_functions.py", line 388, in feature_spatial
    '-V | awk \'{print $1}\''])))
ValueError: invalid literal for int() with base 10: 'Mask and image must be the same size'
Return code: 1

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2

Hi

‘–output-reference-grid’ appears to have been replaced with ‘–template-resample-grid’ in 1.0.8

http://fmriprep.readthedocs.io/en/latest/usage.html

Hope this helps
Bronson

3

I used “–template-resampling-grid 2mm” in the command but it still ended with this error.
Sorry I did not state it clearly during my first post.

Thank you very much!

4

Interesting… It’s a new flavour of a known issue with no resolution at the moment (see https://github.com/poldracklab/fmriprep/issues/870).

As a workaround I would just not use the 2mm resampling grid. There is little reason to use it in context of most applications. You can always reslice your 2mm atlases, use viewers that interpolate data (fsleyes, mango etc.) or extract timeseries taking into account transformation matrices (see BIDS -> fmriprep transforms to (3.5 x 3.75 x 3.75) MNI space? for more details).

If there are some other uses for 2mm I’m missing I would love to hear about them.

5

We are struggling to replicate this on our data. Could you try out this solution? https://github.com/poldracklab/fmriprep/pull/1064#issuecomment-383314377. Thanks in advance.

6

Hi I tried to run it specifying --template MNI152Lin from the bash:

for s in ${sub[@]};do
fmriprep-docker $BIDSfolder $BIDSfolder participant --participant_label ${s} --fs-no-reconall --template MNI152Lin --template-resampling-grid 2mm --use-aroma
done

But it still failed with the error, however, (I didn’t see the --template option listed in the response from the prompt of the terminal, all other options was shown in the response from the terminal, so I am not sure whether I am inputing the option argument in a wrong way?):

180422-02:37:03,534 workflow INFO:
[Node] Running “melodic” (“niworkflows.nipype.interfaces.fsl.model.MELODIC”), a CommandLine Interface with command:
melodic -i /root/src/fmriprep/work/fmriprep_wf/single_subject_001_wf/func_preproc_ses_postdose_task_MID_run_02_wf/ica_aroma_wf/smooth/vol0000_xform-00000_merged_smooth.nii.gz -m /root/src/fmriprep/work/fmriprep_wf/single_subject_001_wf/func_preproc_ses_postdose_task_MID_run_02_wf/bold_mni_trans_wf/mask_mni_tfm/ref_image_corrected_brain_mask_maths_trans.nii.gz --nobet --no_mm -o /root/src/fmriprep
180422-02:45:33,118 workflow WARNING:
[Node] Error on “fmriprep_wf.single_subject_001_wf.func_preproc_ses_postdose_task_rest_wf.ica_aroma_wf.ica_aroma” (/root/src/fmriprep/work/fmriprep_wf/single_subject_001_wf/func_preproc_ses_postdose_task_rest_wf/ica_aroma_wf/ica_aroma)
Traceback (most recent call last):
File “/usr/local/miniconda/bin/fmriprep”, line 11, in
load_entry_point(‘fmriprep==1.0.11’, ‘console_scripts’, ‘fmriprep’)()
File “/usr/local/miniconda/lib/python3.6/site-packages/fmriprep/cli/run.py”, line 274, in main
fmriprep_wf.run(**plugin_settings)
File “/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/workflows.py”, line 602, in run
runner.run(execgraph, updatehash=updatehash, config=self.config)
File “/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/plugins/base.py”, line 168, in run
self._clean_queue(jobid, graph, result=result))
File “/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/plugins/base.py”, line 227, in _clean_queue
raise RuntimeError("".join(result[‘traceback’]))
RuntimeError: Traceback (most recent call last):
File “/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/plugins/multiproc.py”, line 68, in run_node
result[‘result’] = node.run(updatehash=updatehash)
File “/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/nodes.py”, line 487, in run
result = self._run_interface(execute=True)
File “/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/nodes.py”, line 571, in _run_interface
return self._run_command(execute)
File “/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/nodes.py”, line 650, in _run_command
result = self._interface.run(cwd=outdir)
File “/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/interfaces/base/core.py”, line 516, in run
runtime = self._run_interface(runtime)
File “/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/interfaces/report_base.py”, line 51, in _run_interface
ReportCapableInterface, self)._run_interface(runtime)
File “/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/interfaces/base/core.py”, line 1023, in _run_interface
self.raise_exception(runtime)
File “/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/interfaces/base/core.py”, line 960, in raise_exception
).format(**runtime.dictcopy()))
RuntimeError: Command:
ICA_AROMA.py -den nonaggr -i /root/src/fmriprep/work/fmriprep_wf/single_subject_001_wf/func_preproc_ses_postdose_task_rest_wf/ica_aroma_wf/smooth/vol0000_xform-00000_merged_smooth.nii.gz -meldir /root/src/fmriprep/work/fmriprep_wf/single_subject_001_wf/func_preproc_ses_postdose_task_rest_wf/ica_aroma_wf/melodic -mc /root/src/fmriprep/work/fmriprep_wf/single_subject_001_wf/func_preproc_ses_postdose_task_rest_wf/bold_hmc_wf/normalize_motion/motion_params.txt -o /root/src/fmriprep/work/fmriprep_wf/single_subject_001_wf/func_preproc_ses_postdose_task_rest_wf/ica_aroma_wf/ica_aroma
Standard output:

------------------------------- RUNNING ICA-AROMA -------------------------------
--------------- ‘ICA-based Automatic Removal Of Motion Artifacts’ ---------------

Step 1) MELODIC

  • The existing/specified MELODIC directory will be used.
  • The MELODIC directory does not contain the required ‘stats’ folder. Mixture modeling on the Z-statistical maps will be run.
    Step 2) Automatic classification of the components
  • registering the spatial maps to MNI
  • extracting the CSF & Edge fraction features
    • The spatial map of component 1 is empty. Please check!
      Standard error:
      Traceback (most recent call last):
      File “/opt/ICA-AROMA/ICA_AROMA.py”, line 200, in
      edgeFract, csfFract = aromafunc.feature_spatial(fslDir, outDir, scriptDir, melIC_MNI)
      File “/opt/ICA-AROMA/ICA_AROMA_functions.py”, line 388, in feature_spatial
      ‘-V | awk ‘{print $1}’’])))
      ValueError: invalid literal for int() with base 10: ‘Mask and image must be the same size’
      Return code: 1

fMRIPrep: Please report errors to https://github.com/poldracklab/fmriprep/issues

7

Are you using the right version of fmriprep (from the branch in https://github.com/poldracklab/fmriprep/pull/1064). @oesteban maybe it would be easier if you could push a docker image to your docker hub account with this branch for @ChenChiaLan to test?

8

Hi

I am relatively new to using github and fmriprep.
So I think I wasn’t testing it properly, since the code is not in the docker image, is this correct?
Is there anyway to test the pull request locally if using docker to run FMRIPERP?

Thank you very much!

9

Hi @ChenChiaLan,

I’m pushing a docker image for you. This would be your command line modified to work with this development image:

for s in ${sub[@]};do
    fmriprep-docker --image poldracklab/fmriprep:1.0.12-dev0 $BIDSfolder $BIDSfolder participant --participant_label ${s} --fs-no-reconall --template MNI152Lin --template-resampling-grid 2mm --use-aroma
done
1 Like
10

Hi @ChenChiaLan, did the development image fixed your issue?

11

Hi @oesteban

I just finished a previous running batch now, I will test the development image now and let you know as soon as possible.
Thank you very much!

12

Hi @oesteban

I tested the code, this time it ran successfully and created all the image files, tsv, etc.
HOWEVER, I tried to use fsleyes, mricro, mricron, mricroGL and none of them could open the .nii.gz that were generated.
fsleyes showed error loading overlay, cannot work out file type…
mricroGL showed more specific errror: “Warning: the header file is not in NIfTi format [the first 4 bytes do not have the value 348]. Assuming big-endian data.”

Thank you very much!

1 Like
13

Hi @ChenChiaLan,

Thanks very much, you are right about the images not loading. We caught and fixed that problem yesterday (https://github.com/poldracklab/fmriprep/issues/1075).

With your feedback we can merge this fix in and release on Monday.

Thanks a lot!

14

Glad to be of help!

Thank you so much to develop such a robust and excellent pipeline!
Looking forward to the Monday release.

1 Like
15

Hi Experts

After upgrading to 1.0.12, I can run AROMA with 2mm grid but faced another problem.
Sometimes the pipeline will stop with the following error.
Is it to do with my memory setting or some other issue with my docker?

Thank you very much!

180508-02:54:36,135 workflow WARNING:
	 [Node] Error on "fmriprep_wf.single_subject_005_wf.func_preproc_ses_postdose_task_MID_run_02_wf.ica_aroma_wf.ica_aroma" (/root/src/fmriprep/work/fmriprep_wf/single_subject_005_wf/func_preproc_ses_postdose_task_MID_run_02_wf/ica_aroma_wf/ica_aroma)
Traceback (most recent call last):
  File "/usr/local/miniconda/bin/fmriprep", line 11, in <module>
    load_entry_point('fmriprep==1.0.12', 'console_scripts', 'fmriprep')()
  File "/usr/local/miniconda/lib/python3.6/site-packages/fmriprep/cli/run.py", line 314, in main
    fmriprep_wf.run(**plugin_settings)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/workflows.py", line 595, in run
    runner.run(execgraph, updatehash=updatehash, config=self.config)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/plugins/base.py", line 168, in run
    self._clean_queue(jobid, graph, result=result))
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/plugins/base.py", line 227, in _clean_queue
    raise RuntimeError("".join(result['traceback']))
RuntimeError: Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/nodes.py", line 644, in _run_command
    result = self._interface.run(cwd=outdir)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/interfaces/base/core.py", line 520, in run
    runtime = self._run_interface(runtime)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/interfaces/report_base.py", line 69, in _run_interface
    self._generate_report()
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/interfaces/segmentation.py", line 153, in _generate_report
    noise_components_file=self._noise_components_file
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/viz/utils.py", line 489, in plot_melodic_components
    import pylab as plt
  File "/usr/local/miniconda/lib/python3.6/site-packages/pylab.py", line 1, in <module>
    from matplotlib.pylab import *
  File "/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/pylab.py", line 230, in <module>
    import matplotlib.finance
  File "/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/finance.py", line 46, in <module>
    cachedir = get_cachedir()
  File "/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/__init__.py", line 336, in wrapper
    ret = func(*args, **kwargs)
  File "/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/__init__.py", line 692, in _get_cachedir
    return _get_config_or_cache_dir(_get_xdg_cache_dir())
  File "/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/__init__.py", line 657, in _get_config_or_cache_dir
    return _create_tmp_config_dir()
  File "/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/__init__.py", line 577, in _create_tmp_config_dir
    tempdir = tempfile.gettempdir()
  File "/usr/local/miniconda/lib/python3.6/tempfile.py", line 296, in gettempdir
    tempdir = _get_default_tempdir()
  File "/usr/local/miniconda/lib/python3.6/tempfile.py", line 231, in _get_default_tempdir
    dirlist)
FileNotFoundError: [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/root/src/fmriprep/work/fmriprep_wf/single_subject_005_wf/func_preproc_ses_postdose_task_MID_run_02_wf/ica_aroma_wf/ica_aroma']

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/nodes.py", line 480, in run
    result = self._run_interface(execute=True)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/nodes.py", line 564, in _run_interface
    return self._run_command(execute)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/nodes.py", line 648, in _run_command
    _save_resultfile(result, outdir, self.name)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/utils.py", line 246, in save_resultfile
    savepkl(resultsfile, result)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/utils/filemanip.py", line 686, in savepkl
    pkl_file = gzip.open(filename, 'wb')
  File "/usr/local/miniconda/lib/python3.6/gzip.py", line 53, in open
    binary_file = GzipFile(filename, gz_mode, compresslevel)
  File "/usr/local/miniconda/lib/python3.6/gzip.py", line 163, in __init__
    fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
OSError: [Errno 30] Read-only file system: '/root/src/fmriprep/work/fmriprep_wf/single_subject_005_wf/func_preproc_ses_postdose_task_MID_run_02_wf/ica_aroma_wf/ica_aroma/result_ica_aroma.pklz'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/plugins/multiproc.py", line 68, in run_node
    result['result'] = node.run(updatehash=updatehash)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/nodes.py", line 484, in run
    os.remove(hashfile_unfinished)
OSError: [Errno 30] Read-only file system: '/root/src/fmriprep/work/fmriprep_wf/single_subject_005_wf/func_preproc_ses_postdose_task_MID_run_02_wf/ica_aroma_wf/ica_aroma/_0x5ef8f1d1cbe38d826e7ab25181d3afc7_unfinished.json'

Sentry is attempting to send 1 pending error messages
Waiting up to 10 seconds
Press Ctrl-C to quit
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.6/multiprocessing/forkserver.py", line 178, in main
    _serve_one(s, listener, alive_r, handler)
  File "/usr/local/miniconda/lib/python3.6/multiprocessing/forkserver.py", line 212, in _serve_one
    code = spawn._main(child_r)
  File "/usr/local/miniconda/lib/python3.6/multiprocessing/spawn.py", line 115, in _main
    self = reduction.pickle.load(from_parent)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/__init__.py", line 14, in <module>
    import matplotlib
  File "/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/__init__.py", line 1182, in <module>
    rcParams = rc_params()
  File "/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/__init__.py", line 1015, in rc_params
    fname = matplotlib_fname()
  File "/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/__init__.py", line 825, in matplotlib_fname
    configdir = _get_configdir()
  File "/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/__init__.py", line 680, in _get_configdir
    return _get_config_or_cache_dir(_get_xdg_config_dir())
  File "/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/__init__.py", line 657, in _get_config_or_cache_dir
    return _create_tmp_config_dir()
  File "/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/__init__.py", line 577, in _create_tmp_config_dir
    tempdir = tempfile.gettempdir()
  File "/usr/local/miniconda/lib/python3.6/tempfile.py", line 296, in gettempdir
    tempdir = _get_default_tempdir()
  File "/usr/local/miniconda/lib/python3.6/tempfile.py", line 231, in _get_default_tempdir
    dirlist)
FileNotFoundError: [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/root/src/fmriprep']
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.6/multiprocessing/forkserver.py", line 178, in main
    _serve_one(s, listener, alive_r, handler)
  File "/usr/local/miniconda/lib/python3.6/multiprocessing/forkserver.py", line 212, in _serve_one
    code = spawn._main(child_r)
  File "/usr/local/miniconda/lib/python3.6/multiprocessing/spawn.py", line 115, in _main
    self = reduction.pickle.load(from_parent)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/__init__.py", line 14, in <module>
    import matplotlib
  File "/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/__init__.py", line 1182, in <module>
    rcParams = rc_params()
  File "/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/__init__.py", line 1015, in rc_params
    fname = matplotlib_fname()
  File "/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/__init__.py", line 825, in matplotlib_fname
    configdir = _get_configdir()
  File "/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/__init__.py", line 680, in _get_configdir
    return _get_config_or_cache_dir(_get_xdg_config_dir())
  File "/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/__init__.py", line 657, in _get_config_or_cache_dir
    return _create_tmp_config_dir()
  File "/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/__init__.py", line 577, in _create_tmp_config_dir
    tempdir = tempfile.gettempdir()
  File "/usr/local/miniconda/lib/python3.6/tempfile.py", line 296, in gettempdir
    tempdir = _get_default_tempdir()
  File "/usr/local/miniconda/lib/python3.6/tempfile.py", line 231, in _get_default_tempdir
    dirlist)
FileNotFoundError: [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/root/src/fmriprep']
180508-02:54:37,261 workflow WARNING:
	 [Node] Error on "fmriprep_wf.single_subject_005_wf.func_preproc_ses_postdose_task_MID_run_01_wf.ica_aroma_wf.melodic" (/root/src/fmriprep/work/fmriprep_wf/single_subject_005_wf/func_preproc_ses_postdose_task_MID_run_01_wf/ica_aroma_wf/melodic)
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.6/multiprocessing/forkserver.py", line 178, in main
    _serve_one(s, listener, alive_r, handler)
  File "/usr/local/miniconda/lib/python3.6/multiprocessing/forkserver.py", line 212, in _serve_one
    code = spawn._main(child_r)
  File "/usr/local/miniconda/lib/python3.6/multiprocessing/spawn.py", line 115, in _main
    self = reduction.pickle.load(from_parent)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/__init__.py", line 14, in <module>
    import matplotlib
  File "/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/__init__.py", line 1182, in <module>
    rcParams = rc_params()
  File "/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/__init__.py", line 1015, in rc_params
    fname = matplotlib_fname()
  File "/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/__init__.py", line 825, in matplotlib_fname
    configdir = _get_configdir()
  File "/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/__init__.py", line 680, in _get_configdir
    return _get_config_or_cache_dir(_get_xdg_config_dir())
  File "/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/__init__.py", line 657, in _get_config_or_cache_dir
    return _create_tmp_config_dir()
  File "/usr/local/miniconda/lib/python3.6/site-packages/matplotlib/__init__.py", line 577, in _create_tmp_config_dir
    tempdir = tempfile.gettempdir()
  File "/usr/local/miniconda/lib/python3.6/tempfile.py", line 296, in gettempdir
    tempdir = _get_default_tempdir()
  File "/usr/local/miniconda/lib/python3.6/tempfile.py", line 231, in _get_default_tempdir
    dirlist)
FileNotFoundError: [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/root/src/fmriprep']
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.6/multiprocessing/util.py", line 254, in _run_finalizers
    finalizer()
  File "/usr/local/miniconda/lib/python3.6/multiprocessing/util.py", line 186, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/usr/local/miniconda/lib/python3.6/shutil.py", line 474, in rmtree
    _rmtree_safe_fd(fd, path, onerror)
  File "/usr/local/miniconda/lib/python3.6/shutil.py", line 432, in _rmtree_safe_fd
    onerror(os.unlink, fullname, sys.exc_info())
  File "/usr/local/miniconda/lib/python3.6/shutil.py", line 430, in _rmtree_safe_fd
    os.unlink(name, dir_fd=topfd)
OSError: [Errno 30] Read-only file system: 'listener-26fd5vq5'
ERRO[11815] Error waiting for container: driver "overlay2" failed to remove root filesystem for 6c8ccce64cc5717be1722693426116efa9b628ad09ded4e4df0f2ddd207f644a: remove /var/lib/docker/overlay2/ad136cfcd9fc47e4e88b3a92ac0719957673b0ece722e9ef2fedcec8e3f0e2dc/diff/out: read-only file system 
fMRIPrep: Please report errors to https://github.com/poldracklab/fmriprep/issues
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/0f88cf74aef455a07eb67b57cb59c3fbd4a9ac821f4d65dbe1193717ee39111a-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/4ced0a5b510e8e1b429f73c4bbb7674607e769e5529c33a842d55fa047c2912e-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/bd3e6395094ee40ad0f99ba8b28095374c1fa329381627a41d63688f3d286c21-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/15d8085ffa6668d16b7e32d7d9e4ff9ed72343a947b5cde36db2a2991e0f3ec1-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/7e459ccd2e362cb187d8e867550f842512f1ece03b53a83753b934ad5e9ef924-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/fd4070f5c79583e8688fc06d3d0c5b5723e0c99400c71f20dae0f17708151889-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/6454ef1efb4080ef3c276886eae76b57161d330a058af9eedfc8926af43f6242-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/b371ab2b2676543b0fd07e6089152c19e72055cb5ff5dffd7e185273e4df8fff-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/1811197f2952951432e2815167ffee47f3f12e79e50f1ba27524b915e21ce1f8-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/51146d6659b56f6b98e7f33c19c76d47a68fc8bce54acf77ccb4840f9ffe2519-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/319e44c33e434583810761b9d465bf8b5be2e62b809bdba133bad567a9f2d8b9-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/56e97bee14fae7e5996e482627d5033e7b11d97b81fd2460076296de7221f297-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/28c1f23c5fcef5b1f2d5fc80b8c397e5f364eff096210817dc9059fb614ec171-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/7133d20c0423f69679b212cadab60977a3e96819ebcd4de3be34f82c413812af-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/e45552fe490c9a2748ddcceed81a385d1f0d3dfee24f46a9aeb47fe262f92242-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/8aca2c5b3a8e28e0c20fa98a0f08085a683139cec2ddf6fa40a50d6185447fc6-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/6ec33b6eaa07aef8d03e54f0a86a1a4b96d1a4df679b259f6c77322d86d30c2c-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/0085dbb0c5cb1df6f214a6fa460abe4e19da7f641543ba742911ae8573efd94b-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/299870502be215a79bd426dd1feb3bc0c2a00a5b83120269f8388b82831943d7-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/3382d0a63787f93674d2349c2c5af4a7d19235342c76bc3a96d7e00aecde806b-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/bc4d0007b7fc008a234981eef1b6ca4dabd06e75f64c33df1d1517acfeb42fad-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/3b99560ca075a1669f0953365647614d87e9abfb3b9721aca517be13520cca55-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/aaa2ba73421616cdb520b34f0387621fe77dd2f69ba1a4189bc7264cf435437b-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/2154a4a61843195af36dbb58e0de034857ec894742576bf8a448d52dd7546305-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/accd5bfe2dd242645169c8834f0e5692ff02f5635104ed64c895cc3b58c3f524-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/2f8dbfd1c6dc15bf1da479f9ffb9440fedc4fd6de5671f289fd5a804f2ea161f-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/0b63efd3c0093f184d3f43b2f089e31004a3e524bc3c9509b7119ac116da0e47-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/10d9aaaac3833764f776f5fb3ff8b96b6ff53e032665ffa789ea91fd46d58367-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/cb2c51a20d6b59b38f32bcd5b70d133ef8c73235c77605ab3dd1d91b18edb146-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
docker: Error response from daemon: mkdir /var/lib/docker/overlay2/0f700579ad76df51af2f2d5d8f6c515d61056c483a17c22da65024de643f815c-init: read-only file system.
See 'docker run --help'.
Could not detect memory capacity of Docker container.
Do you have permission to run docker?
16

This looks like it may be a disk space or quota issue, and it’s crashing Docker. If you run df -h are any of your partitions full? And what output do you see when you run quota and sudo quota?

17

Hi

For df -h I got:
Filesystem Size Used Avail Capacity iused ifree %iused Mounted on
/dev/disk1s1 1.8Ti 1.7Ti 136Gi 93% 3592046 9223372036851183761 0% /
devfs 196Ki 196Ki 0Bi 100% 678 0 100% /dev
/dev/disk1s4 1.8Ti 11Gi 136Gi 8% 11 9223372036854775796 0% /private/var/vm
map -hosts 0Bi 0Bi 0Bi 100% 0 0 100% /net
map auto_home 0Bi 0Bi 0Bi 100% 0 0 100% /home

And for quota and sudo quota, I got both response: none

Thank you!

18

Okay, so at least your host system has plenty of space. My guess, then, is that there are Docker-related constraints at play. Rather than try to debug these, I would consider using -w <somedir> to mount the scratch directory from your host system into Docker. It could be something like /tmp/fmriprep_wd.

Note that this space will not be freed when processing completes, and you’ll almost certainly want to use a different working directory for each dataset that you process (not necessarily each subject, though you’ll want to be careful about running multiple jobs in parallel; please ask for advice if you plan to do this).