Docker & fMRIPrep: PermissionError: [Errno 13] Permission denied: '/.cache'`

pvelasco · February 20, 2019, 4:38pm

Hi,
Not sure if the following is relevant for this topic. I’m having a similar error when running the official docker images (from Dockerhub) from version 1.3.0 on (v 1.2.6-1 runs fine):

My call:

docker run --name test_fmriprep \
           -e "FS_LICENSE=/FSLicenseFolder/$fsLicenseBasename" \
           --user $userID \
           --rm \
           --volume /CBI/UserData:/data \
           --volume ${fsLicenseFolder}:/FSLicenseFolder:ro \
           poldracklab/fmriprep:1.3.0 \
               /data/${labDir} \
               /data/${labDir}/derivatives \
               participant \
               --participant_label ${subjectID}

File "/usr/local/miniconda/lib/python3.7/site-packages/datalad/utils.py", line 1198, in assure_dir
    os.makedirs(dirname)
[...]
PermissionError: [Errno 13] Permission denied: '/.cache'`

When I modify datalad/utils.py to print out dirname, it prints:

/.cache/datalad/sockets

The problem might be that I’m running the docker container as a regular user, not as root. As such, I don’t have writing privileges at the root directory (/).

Maybe datalad should try to write its cache in some other folder?

Thanks,

-Pablo

oesteban · February 20, 2019, 7:26pm

Hi @pvelasco

Can you give this a try?:

docker run --name test_fmriprep \
           -e "TEMPLATEFLOW_HOME=/opt/templateflow" \
           -e "FS_LICENSE=/FSLicenseFolder/$fsLicenseBasename" \
           --user $userID \
           --rm \
           --volume /CBI/UserData:/data \
           --volume ${fsLicenseFolder}:/FSLicenseFolder:ro \
           poldracklab/fmriprep:1.3.0 \
               /data/${labDir} \
               /data/${labDir}/derivatives \
               participant \
               --participant_label ${subjectID}

(please note how I’m setting the TEMPLATEFLOW_HOME env variable).

pvelasco · February 21, 2019, 2:15pm

Hi @oesteban

Thanks for the suggestion.

Still, the same problem. Here is the log, after the BIDS validation:

190221-14:12:12,872 nipype.workflow IMPORTANT:
	 
    Running fMRIPREP version 1.3.0.post1:
      * BIDS dataset path: /data/phelpslab/Linda/BIDSdata.
      * Participant list: ['Pilot005'].
      * Run identifier: 20190221-141212_7eb4fa3b-3b9e-4ad1-bfda-a62b97208504.
    
/usr/local/miniconda/lib/python3.7/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__
  return f(*args, **kwds)
/usr/local/miniconda/lib/python3.7/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__
  return f(*args, **kwds)
/usr/local/miniconda/lib/python3.7/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__
  return f(*args, **kwds)
/usr/local/miniconda/lib/python3.7/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__
  return f(*args, **kwds)
/usr/local/miniconda/lib/python3.7/site-packages/nilearn/datasets/neurovault.py:16: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  from collections import Container
/usr/local/miniconda/lib/python3.7/site-packages/datalad/utils.py:71: DeprecationWarning: dist() and linux_distribution() functions are deprecated in Python 3.5
  = platform.linux_distribution()[:2]
/usr/local/miniconda/lib/python3.7/site-packages/datalad/utils.py:847: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  is_decorating = not kwargs and len(args) == 1 and isinstance(args[0], collections.Callable)
/usr/local/miniconda/lib/python3.7/site-packages/datalad/support/param.py:30: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()
  _KNOWN_ARGS = getargspec(argparse.Action.__init__)[0] + ['action']
/usr/local/miniconda/lib/python3.7/site-packages/datalad/interface/base.py:216: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()
  args, varargs, varkw, defaults = getargspec(func)
/usr/local/miniconda/lib/python3.7/site-packages/skimage/__init__.py:80: ResourceWarning: unclosed file <_io.TextIOWrapper name='/usr/local/miniconda/lib/python3.7/site-packages/pytest.py' mode='r' encoding='utf-8'>
  imp.find_module('pytest')
/usr/local/miniconda/lib/python3.7/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__
  return f(*args, **kwds)
/usr/local/miniconda/lib/python3.7/site-packages/networkx/classes/reportviews.py:95: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  from collections import Mapping, Set, Iterable
/usr/local/miniconda/lib/python3.7/site-packages/datalad/interface/base.py:480: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()
  argspec = getargspec(call)
Process Process-2:
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/cli/run.py", line 751, in build_workflow
    ignore_aroma_err=opts.ignore_aroma_denoising_errors,
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/workflows/base.py", line 218, in init_fmriprep_wf
    ignore_aroma_err=ignore_aroma_err,
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/workflows/base.py", line 461, in init_single_subject_wf
    name="anat_preproc_wf",
  File "/usr/local/miniconda/lib/python3.7/site-packages/smriprep/workflows/anatomical.py", line 171, in init_anat_preproc_wf
    template_meta = get_metadata(template)
  File "/usr/local/miniconda/lib/python3.7/site-packages/templateflow/api.py", line 70, in get_metadata
    api.get(str(filepath))
  File "/usr/local/miniconda/lib/python3.7/site-packages/datalad/interface/utils.py", line 478, in eval_func
    return return_func(generator_func)(*args, **kwargs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/datalad/interface/utils.py", line 466, in return_func
    results = list(results)
  File "/usr/local/miniconda/lib/python3.7/site-packages/datalad/interface/utils.py", line 421, in generator_func
    result_renderer, result_xfm, _result_filter, **_kwargs):
  File "/usr/local/miniconda/lib/python3.7/site-packages/datalad/interface/utils.py", line 490, in _process_results
    for res in results:
  File "/usr/local/miniconda/lib/python3.7/site-packages/datalad/distribution/get.py", line 610, in __call__
    if not isinstance(ds.repo, AnnexRepo):
  File "/usr/local/miniconda/lib/python3.7/site-packages/datalad/distribution/dataset.py", line 217, in repo
    self._repo = cls(self._path, create=False, **kw)
  File "/usr/local/miniconda/lib/python3.7/site-packages/datalad/support/repo.py", line 146, in __call__
    instance = type.__call__(cls, *new_args, **new_kwargs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/datalad/support/annexrepo.py", line 187, in __init__
    fake_dates=fake_dates)
  File "/usr/local/miniconda/lib/python3.7/site-packages/datalad/support/gitrepo.py", line 538, in __init__
    ssh_manager.assure_initialized()
  File "/usr/local/miniconda/lib/python3.7/site-packages/datalad/support/sshconnector.py", line 376, in assure_initialized
    assure_dir(self._socket_dir)
  File "/usr/local/miniconda/lib/python3.7/site-packages/datalad/utils.py", line 1198, in assure_dir
    os.makedirs(dirname)
  File "/usr/local/miniconda/lib/python3.7/os.py", line 211, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/local/miniconda/lib/python3.7/os.py", line 211, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/local/miniconda/lib/python3.7/os.py", line 221, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/.cache'

Any other ideas?

Thanks,

-Pablo

oesteban · February 21, 2019, 2:29pm

Thanks, I’ll need to take a closer look.

oesteban · February 22, 2019, 6:16am

Hi @pvelasco,

What you are seeing is derived from running docker with --user $userID. The problem is that, when mapping users by UID and the UID is not defined in the container then $HOME is set to / by the runtime.

Before the patch is rolled out, you can also try to remap the user to root with any of these options: https://docs.docker.com/engine/security/userns-remap/

oesteban · February 22, 2019, 8:35am

Found a workaround while we roll out a new release:

Create a directory, eg.:

mkdir -p $HOME/tmp/home

Run your command line, mounting it and setting HOME:

docker run --name test_fmriprep \
           -v $HOME/tmp/home:/home/user -e "HOME=/home/user" \
           -e "FS_LICENSE=/FSLicenseFolder/$fsLicenseBasename" \
           --user $userID \
           --rm \
           --volume /CBI/UserData:/data \
           --volume ${fsLicenseFolder}:/FSLicenseFolder:ro \
           poldracklab/fmriprep:1.3.0.post2 \
               /data/${labDir} \
               /data/${labDir}/derivatives \
               participant \
               --participant_label ${subjectID}

I’ve tested this with 1.3.0.post2 and worked.

oesteban · February 22, 2019, 7:25pm

@pvelasco

Could you check whether the docker image I just built works for you?

docker run --name test_fmriprep \
           -e "FS_LICENSE=/FSLicenseFolder/$fsLicenseBasename" \
           --user $userID \
           --rm \
           --volume /CBI/UserData:/data \
           --volume ${fsLicenseFolder}:/FSLicenseFolder:ro \
           poldracklab/fmriprep:user-home-test \
               /data/${labDir} \
               /data/${labDir}/derivatives \
               participant \
               --participant_label ${subjectID}

oesteban · March 1, 2019, 5:26pm

Hi @pvelasco,

Release 1.3.0.post3 should account for this issue and further problems you’ve reported via GitHub. Please let us know whether this command worked for you:

docker run --name test_fmriprep \
           -e "FS_LICENSE=/FSLicenseFolder/$fsLicenseBasename" \
           --user $userID \
           --rm \
           --volume /CBI/UserData:/data \
           --volume ${fsLicenseFolder}:/FSLicenseFolder:ro \
           poldracklab/fmriprep:1.3.0.post3 \
               /data/${labDir} \
               /data/${labDir}/derivatives \
               participant \
               --participant_label ${subjectID}

oesteban · March 6, 2019, 8:46pm

Hi, the latest release 1.3.1 is out. Please let us know if that version resolves this problem!

pvelasco · March 7, 2019, 2:24pm

Yes, it does. Thanks!

a3sha2 · May 14, 2019, 3:21pm

How to specify template flow for singularity image

oesteban · May 14, 2019, 6:46pm

You’d need to export it before the fmriprep call, and avoid using --cleanenv:

export TEMPLATEFLOW_HOME=/opt/templateflow
unset PYTHONPATH
singularity run fmriprep.img ...

the unset is just to be sure.