Problem using neurodocker and singularity-hub

Hi all,

I’m trying to use neurodocker via singularity to build a singularity recipe which I would like to build on singularity-hub and then download. The issue I am running into is that after building on singularity-hub, the container is missing my conda packages and I see log lines as follows:

Will remove the following tarballs:
/opt/miniconda-latest/pkgs
--------------------------
...
idna-2.8-py37_0.conda 85 KB
openssl-1.1.1e-h7b6447c_0.conda 2.5 MB
---------------------------------------------------
Total: 76.6 MB
Proceed ([y]/n)? Invalid choice: conda create -y -q --name neuro
Proceed ([y]/n)? Invalid choice: conda install -y -q --name neuro \
Proceed ([y]/n)? Invalid choice: "numpy" \

This suggests to me that it is not answering this prompt and thus not installing any Conda packages.

I’ve tried to modify my recipe to echo “y” there but that did not work. This is my first time using all of these programs, am I going about this the wrong way or has anyone run into similar issues? Any help or input would be appreciated, thanks!


More detailed info:

I use the following neurodocker command (using singularity):

singularity run neurodocker.simg generate singularity \
--base centos:7 \
--pkg-manager yum \
--miniconda create_env=neuro \
conda_install="numpy pandas scipy scikit-learn matplotlib seaborn networkx ipython joblib nibabel datalad" \
pip_install="nibabel ply neurosynth nilearn pliers" \
--install emacs > Singularity

This seems to work and gives me a recipe:

# Your version: 0.6.0 Latest version: 0.6.0
# Generated by Neurodocker version 0.6.0
# Timestamp: 2020-03-20 20:25:56 UTC
# 
# Thank you for using Neurodocker. If you discover any issues
# or ways to improve this software, please submit an issue or
# pull request on our GitHub repository:
# 
#     https://github.com/kaczmarj/neurodocker

Bootstrap: docker
From: centos:7

%post
export ND_ENTRYPOINT="/neurodocker/startup.sh"
yum install -y -q \
    bzip2 \
    ca-certificates \
    curl \
    localedef \
    unzip
yum clean packages
rm -rf /var/cache/yum/*
localedef -i en_US -f UTF-8 en_US.UTF-8
chmod 777 /opt && chmod a+s /opt
mkdir -p /neurodocker
if [ ! -f "$ND_ENTRYPOINT" ]; then
  echo '#!/usr/bin/env bash' >> "$ND_ENTRYPOINT"
  echo 'set -e' >> "$ND_ENTRYPOINT"
  echo 'export USER="${USER:=`whoami`}"' >> "$ND_ENTRYPOINT"
  echo 'if [ -n "$1" ]; then "$@"; else /usr/bin/env bash; fi' >> "$ND_ENTRYPOINT";
fi
chmod -R 777 /neurodocker && chmod a+s /neurodocker

export PATH="/opt/miniconda-latest/bin:$PATH"
echo "Downloading Miniconda installer ..."
conda_installer="/tmp/miniconda.sh"
curl -fsSL --retry 5 -o "$conda_installer" https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash "$conda_installer" -b -p /opt/miniconda-latest
rm -f "$conda_installer"
conda update -yq -nbase conda
conda config --system --prepend channels conda-forge
conda config --system --set auto_update_conda false
conda config --system --set show_channel_urls true
sync && conda clean --all && sync
echo 'y'
conda create -y -q --name neuro
conda install -y -q --name neuro \
    "numpy" \
    "pandas" \
    "scipy" \
    "scikit-learn" \
    "matplotlib" \
    "seaborn" \
    "networkx" \
    "ipython" \
    "joblib" \
    "nibabel" \
    "datalad"
sync && conda clean --all && sync
bash -c "source activate neuro
  pip install --no-cache-dir  \
      "nibabel" \
      "ply" \
      "neurosynth" \
      "nilearn" \
      "pliers""
rm -rf ~/.cache/pip/*
sync


yum install -y None \
    emacs
yum clean packages
rm -rf /var/cache/yum/*

echo '{
\n  "pkg_manager": "yum",
\n  "instructions": [
\n    [
\n      "base",
\n      "centos:7"
\n    ],
\n    [
\n      "_header",
\n      {
\n        "version": "generic",
\n        "method": "custom"
\n      }
\n    ],
\n    [
\n      "miniconda",
\n      {
\n        "create_env": "neuro",
\n        "conda_install": [
\n          "numpy",
\n          "pandas",
\n          "scipy",
\n          "scikit-learn",
\n          "matplotlib",
\n          "seaborn",
\n          "networkx",
\n          "ipython",
\n          "joblib",
\n          "nibabel",
\n          "datalad"
\n        ],
\n        "pip_install": [
\n          "nibabel",
\n          "ply",
\n          "neurosynth",
\n          "nilearn",
\n          "pliers"
\n        ]
\n      }
\n    ],
\n    [
\n      "install",
\n      [
\n        "emacs"
\n      ]
\n    ]
\n  ]
\n}' > /neurodocker/neurodocker_specs.json

%environment
export LANG="en_US.UTF-8"
export LC_ALL="en_US.UTF-8"
export ND_ENTRYPOINT="/neurodocker/startup.sh"
export CONDA_DIR="/opt/miniconda-latest"
export PATH="/opt/miniconda-latest/bin:$PATH"

%runscript
/neurodocker/startup.sh "$@"

I’m not sure right now why this is not working. But it seems this is specific to centos and singularity. I was able to generate a Dockerfile using a neurodocker specification you provided and build a docker container without any issue.

For the singularity image, I’ve checked that I’m able to create an image if the base one is debian. Not sure if that helps… We can search more, or perhaps someone else knows the answer

Ah thanks for checking! I tried this also earlier with debian stretch as the base OS and it seemed to build fine. The issue I then had was that I could not use any modules still. Upon starting python I could not load numpy for example.

It could be a problem with activating the conda env:

Singularity> conda info --envs
# conda environments:
#
base                  *  /opt/miniconda-latest
neuro                    /opt/miniconda-latest/envs/neuro

Singularity> activate neuro
Singularity> conda activate neuro

CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run

    $ conda init <SHELL_NAME>

Currently supported shells are:
  - bash
  - fish
  - tcsh
  - xonsh
  - zsh
  - powershell

See 'conda init --help' for more information and options.

IMPORTANT: You may need to close and restart your shell after running 'conda init'.

Though from the documentation I thought this would be activated by default. Haven’t yet been able to figure this out.

perhaps something is out of date in neurodocker, maybe you can open an issue, to remember that we should check this.

if you want to activate once you are inside, you can run source activate neuro.

Or try to run conda init bash (or other shells), you will have to read from the output which files has been modified by conda and run source ..../.bashrc (or something similar)

1 Like

Thanks, that did it! I will open an issue on neurodocker re: centos / singularity problem

I am running into a related problem: source activate neuro was working great for running my singularity image interactively but I can’t activate my conda env via a bash script (for running things on the cluster). This seems to be a general problem for people online but none of the solutions work for using source activate as opposed to conda activate and I still can’t get conda activate to work. Is there an easy work-around for this?

When I enter conda init bash it lists no changes and No action taken so I can’t find any specific path to run source

singularity run uses the init script but singularity shell and exec do not. could that be the reason?

I don’t think so since I’ve only used singularity shell and exec with this container not run. In the slurm script that I use to call this I was using shell and I get the error:

line 11: activate: No such file or directory

If I try to use singularity exec to only activate the env I get this:

bash-4.2$ singularity exec image.sif source activate neuro

/.singularity.d/actions/exec: line 21: exec: source: not found

is your container publicly available somewhere?

Yup it is available here!

In the past, I was trying to get neurodocker to generate the recipe such that it would activate the env as it started up but couldn’t quite get it to work at the time, maybe that will be a better solution than activating it each time it starts.