Preferred way to create MRIQC singularity container?

,
1

I’m trying to create a singularity container with MRIQC pulling directly from docker. The image gets created without problems, however MRIQC cannot be run because of incorrect permissions for miniconda. I could create a singularity definition file for MRIQC, but I wonder if there’s an easier solution in this case.

EDIT: I meant preferred way without using docker2singularity. Not because of any particular reason, but because the docker2singularity’s README says:

Note that the upcoming Singularity (2.3) release supports import from Docker without sudo, natively, and this is the recommended approach

Here’s how I created the singularity container with singularity 2.3.2

mvdoc@euclid ~/singularity $ singularity --version
2.3.2-heads/2.3.2.gbf5853e
mvdoc@euclid ~/singularity $ sudo singularity pull --size 8000 docker://poldracklab/mriqc:latest

EDIT2: Running singularity pull without sudo results in the same permission error.

and when running the image this is what happens

mvdoc@euclid ~/singularity $ singularity run -e -C mriqc-latest.img
Traceback (most recent call last):
  File "/usr/local/miniconda/bin/mriqc", line 6, in <module>
    from pkg_resources import load_entry_point
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 646, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 616, in _load_backward_compatible
  File "/usr/local/miniconda/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/__init__.py", line 2985, in <module>
  File "/usr/local/miniconda/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/__init__.py", line 2971, in _call_aside
  File "/usr/local/miniconda/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/__init__.py", line 2998, in _initialize_master_working_set
  File "/usr/local/miniconda/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/__init__.py", line 651, in _build_master
  File "/usr/local/miniconda/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/__init__.py", line 644, in __init__
  File "/usr/local/miniconda/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/__init__.py", line 700, in add_entry
  File "/usr/local/miniconda/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/__init__.py", line 1990, in find_on_path
  File "/usr/local/miniconda/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/__init__.py", line 2399, in from_location
  File "/usr/local/miniconda/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/__init__.py", line 2740, in _reload_version
  File "/usr/local/miniconda/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/__init__.py", line 2364, in _version_from_file
  File "/usr/local/miniconda/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/__init__.py", line 2532, in _get_metadata
  File "/usr/local/miniconda/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/__init__.py", line 1873, in get_metadata_lines
  File "/usr/local/miniconda/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/pkg_resources/__init__.py", line 1861, in get_metadata
PermissionError: [Errno 13] Permission denied: '/usr/local/miniconda/lib/python3.6/site-packages/.wh.conda-4.3.11-py3.6.egg-info'

And everything in /usr/local/miniconda is owned by root

mvdoc@euclid ~/singularity $ singularity exec mriqc-latest.img ls -l /usr/local/miniconda
total 140
-rw-rw-r--  1 root root  3699 Feb  4  2016 LICENSE.txt
drwxr-xr-x  2 root root 12288 Oct 10 06:21 bin
drwxr-xr-x  2 root root  4096 Oct 10 06:23 conda-bld
drwxr-xr-x  2 root root  4096 Oct 10 06:21 conda-meta
drwxr-xr-x  3 root root  4096 Oct 10 06:23 doc
drwxr-xr-x  2 root root  4096 Oct 10 06:21 envs
drwxr-xr-x  6 root root  4096 Oct 10 06:21 etc
drwxr-xr-x 23 root root 12288 Oct 10 06:21 include
drwxr-xr-x 14 root root 36864 Oct 10 06:21 lib
drwxr-xr-x  4 root root  4096 Oct 10 06:23 libexec
drwxr-xr-x  3 root root  4096 Oct 10 06:23 man
drwxr-xr-x 97 root root  4096 Oct 10 06:23 mkspecs
drwxr-xr-x  2 root root  4096 Oct 10 06:23 phrasebooks
drwxr-xr-x 97 root root  4096 Oct 10 06:21 pkgs
drwxr-xr-x 16 root root  4096 Oct 10 06:23 plugins
drwxr-xr-x 14 root root  4096 Oct 10 06:23 qml
drwxr-xr-x 19 root root  4096 Oct 10 06:21 share
drwxr-xr-x  3 root root  4096 Oct 10 06:21 ssl
drwxr-xr-x  2 root root 12288 Oct 10 06:23 translations
drwxr-xr-x  3 root root  4096 Oct 10 06:23 var
drwxr-xr-x  3 root root  4096 Oct 10 06:23 x86_64-conda_cos6-linux-gnu

thanks!

2

This is most likely an issue with how singularity is performing permission changes. Please raise it at https://github.com/singularityware/singularity/issues if it is not already discussed.

While docker support matures in singularity I would recommend docker2singularity as a fallback.

3

Thanks. docker2singularity did the trick for now. And I created an issue to follow up on this.

4

those .wh files are aufs whiteout files that get created during docker creation (nothing to do with conda, but with file sync post conda). we will soon be switching nipype’s builder to neurodocker, which takes care of ensuring those files are not preserved in any layer. switching nipype should impact the mriqc and fmriprep containers to not have those files.

the current issue has to do with the docker image itself rather than the conversion. https://github.com/singularityware/singularity/issues/571 (you have to read through a fair bit to get to this). the reason docker2singularity works is that docker realizes these as useless cache files and removes them from the container when rebuilding and exporting as an image.

2 Likes