Hello,
I’d previously gotten a custom Neurodocker image working, which opened as a Jupyter Notebook, but since had to clear my docker images.
What is strange is that the build worked on Monday (02/19/2018), but after having to clear my docker images and rebuild again, it would no longer work on Wednesday (02/21/2018). I’ve tried to debug this, reseting Docker to factory defaults, trying a different Neurodebian build (trust-non-free, instead of stretch-non-free as I’ve been using), but these have not solved it. I’m wondering if there may have been some update this week that broke compatibility?
Anyway, I’m posting in case anyone has thoughts on how to go about debugging this build. The problem appears to stem from this command in the Dockerfile.
#-------------------------
# Create conda environment
#-------------------------
RUN conda create -y -q --name neuro python=3.6 \
jupyter \
jupyterlab \
jupyter_contrib_nbextensions \
traits \
pandas \
matplotlib \
scikit-learn \
seaborn \
&& sync && conda clean -tipsy && sync \
&& /bin/bash -c "source activate neuro \
&& pip install -q --no-cache-dir https://github.com/nipy/nipype/tarball/master \
https://github.com/INCF/pybids/tarball/master \
nilearn \
datalad[full] \
nipy \
duecredit" \
&& sync \
&& sed -i '$isource activate neuro' $ND_ENTRYPOINT
Here is the neurodocker command to build the Dockerfile:
docker run --rm kaczmarj/neurodocker:master generate -b neurodebian:stretch-non-free -p apt \
--install convert3d ants fsl gcc g++ graphviz tree \
git-annex-standalone vim emacs-nox nano less ncdu \
tig git-annex-remote-rclone \
--freesurfer version=6.0.0 min=true \
--afni version=latest \
--add-to-entrypoint "source /etc/fsl/fsl.sh" \
--spm version=12 matlab_version=R2017a \
--user=neuro \
--miniconda \
conda_install="python=3.6 jupyter jupyterlab jupyter_contrib_nbextensions
traits pandas matplotlib scikit-learn seaborn" \
pip_install="https://github.com/nipy/nipype/tarball/master
https://github.com/INCF/pybids/tarball/master
nilearn datalad[full] nipy duecredit" \
env_name="neuro" \
activate=True \
--run-bash "source activate neuro && jupyter nbextension enable exercise2/main && jupyter nbextension enable spellchecker/main" \
--run 'mkdir -p ~/.jupyter && echo c.NotebookApp.ip = \"0.0.0.0\" > ~/.jupyter/jupyter_notebook_config.py' \
--user=root \
--miniconda \
pip_install="git+https://github.com/poldracklab/fmriprep.git
git+https://github.com/poldracklab/niworkflows.git" \
env_name="neuro" \
activate=True \
--run 'chown -R neuro /home/neuro' \
--run 'mkdir /data && chmod 777 /data && chmod a+s /data' \
--run 'mkdir /output && chmod 777 /output && chmod a+s /output' \
--user=neuro \
--workdir /home/neuro \
--cmd "jupyter-notebook" \
--no-check-urls > Dockerfile
And here is the full output of the build attempt from the resulting Dockerfile (with successful steps cached):
$ docker build --rm -t my_nipype .
Sending build context to Docker daemon 44.03kB
Step 1/30 : FROM neurodebian:stretch-non-free
---> 0f99b7ca24a5
Step 2/30 : ARG DEBIAN_FRONTEND=noninteractive
---> Using cache
---> 2215f5ffcff6
Step 3/30 : ENV LANG="en_US.UTF-8" LC_ALL="C.UTF-8" ND_ENTRYPOINT="/neurodocker/startup.sh"
---> Using cache
---> ae4bcceb79be
Step 4/30 : RUN apt-get update -qq && apt-get install -yq --no-install-recommends apt-utils bzip2 ca-certificates curl locales unzip && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* && localedef --force --inputfile=en_US --charmap=UTF-8 C.UTF-8 && chmod 777 /opt && chmod a+s /opt && mkdir -p /neurodocker && if [ ! -f "$ND_ENTRYPOINT" ]; then echo '#!/usr/bin/env bash' >> $ND_ENTRYPOINT && echo 'set +x' >> $ND_ENTRYPOINT && echo 'if [ -z "$*" ]; then /usr/bin/env bash; else $*; fi' >> $ND_ENTRYPOINT; fi && chmod -R 777 /neurodocker && chmod a+s /neurodocker
---> Using cache
---> ff3a7d1c0d71
Step 5/30 : ENTRYPOINT ["/neurodocker/startup.sh"]
---> Using cache
---> cf803b825a9c
Step 6/30 : RUN apt-get update -qq && apt-get install -y -q --no-install-recommends convert3d ants fsl gcc g++ graphviz tree git-annex-standalone vim emacs-nox nano less ncdu tig git-annex-remote-rclone && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
---> Using cache
---> d68b72540668
Step 7/30 : RUN apt-get update -qq && apt-get install -yq --no-install-recommends bc libgomp1 libxmu6 libxt6 tcsh perl && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* && echo "Downloading minimized FreeSurfer ..." && curl -sSL --retry 5 https://dl.dropbox.com/s/nnzcfttc41qvt31/recon-all-freesurfer6-3.min.tgz | tar xz -C /opt && sed -i '$isource $FREESURFER_HOME/SetUpFreeSurfer.sh' $ND_ENTRYPOINT
---> Using cache
---> 1bdb11f8da18
Step 8/30 : ENV FREESURFER_HOME=/opt/freesurfer
---> Using cache
---> 653ecdafbcc3
Step 9/30 : ENV PATH=/opt/afni:$PATH
---> Using cache
---> 1026cf261191
Step 10/30 : RUN apt-get update -qq && apt-get install -yq --no-install-recommends ed gsl-bin libglu1-mesa-dev libglib2.0-0 libglw1-mesa libgomp1 libjpeg62 libxm4 netpbm tcsh xfonts-base xvfb && libs_path=/usr/lib/x86_64-linux-gnu && if [ -f $libs_path/libgsl.so.19 ]; then ln $libs_path/libgsl.so.19 $libs_path/libgsl.so.0; fi && echo "Install libxp (not in all ubuntu/debian repositories)" && apt-get install -yq --no-install-recommends libxp6 || /bin/bash -c " curl --retry 5 -o /tmp/libxp6.deb -sSL http://mirrors.kernel.org/debian/pool/main/libx/libxp/libxp6_1.0.2-2_amd64.deb && dpkg -i /tmp/libxp6.deb && rm -f /tmp/libxp6.deb" && echo "Install libpng12 (not in all ubuntu/debian repositories" && apt-get install -yq --no-install-recommends libpng12-0 || /bin/bash -c " curl --retry 5 -o /tmp/libpng12.deb -sSL http://mirrors.kernel.org/debian/pool/main/libp/libpng/libpng12-0_1.2.49-1%2Bdeb7u2_amd64.deb && dpkg -i /tmp/libpng12.deb && rm -f /tmp/libpng12.deb" && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* && echo "Downloading AFNI ..." && mkdir -p /opt/afni && curl -sSL --retry 5 https://afni.nimh.nih.gov/pub/dist/tgz/linux_openmp_64.tgz | tar zx -C /opt/afni --strip-components=1
---> Using cache
---> 8bdfdcec3d95
Step 11/30 : RUN sed -i '$isource /etc/fsl/fsl.sh' $ND_ENTRYPOINT
---> Using cache
---> c0713ba561e0
Step 12/30 : RUN apt-get update -qq && apt-get install -yq --no-install-recommends libxext6 libxt6 && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* && echo "Downloading MATLAB Compiler Runtime ..." && curl -sSL --retry 5 -o /tmp/mcr.zip https://www.mathworks.com/supportfiles/downloads/R2017a/deployment_files/R2017a/installers/glnxa64/MCR_R2017a_glnxa64_installer.zip && unzip -q /tmp/mcr.zip -d /tmp/mcrtmp && /tmp/mcrtmp/install -destinationFolder /opt/mcr -mode silent -agreeToLicense yes && rm -rf /tmp/*
---> Using cache
---> 63363c20146d
Step 13/30 : RUN echo "Downloading standalone SPM ..." && curl -sSL --retry 5 -o spm.zip http://www.fil.ion.ucl.ac.uk/spm/download/restricted/utopia/dev/spm12_latest_Linux_R2017a.zip && unzip -q spm.zip -d /opt && chmod -R 777 /opt/spm* && rm -rf spm.zip && /opt/spm12/run_spm12.sh /opt/mcr/v92/ quit && sed -i '$iexport SPMMCRCMD=\"/opt/spm12/run_spm12.sh /opt/mcr/v92/ script\"' $ND_ENTRYPOINT
---> Using cache
---> b3b0729bff45
Step 14/30 : ENV MATLABCMD=/opt/mcr/v92/toolbox/matlab FORCE_SPMMCR=1 LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu:/opt/mcr/v92/runtime/glnxa64:/opt/mcr/v92/bin/glnxa64:/opt/mcr/v92/sys/os/glnxa64:$LD_LIBRARY_PATH
---> Using cache
---> 8e29826f7fba
Step 15/30 : RUN useradd --no-user-group --create-home --shell /bin/bash neuro
---> Using cache
---> 679cc11094f9
Step 16/30 : USER neuro
---> Using cache
---> 700551d8361b
Step 17/30 : ENV CONDA_DIR=/opt/conda PATH=/opt/conda/bin:$PATH
---> Using cache
---> 46b98eabad4c
Step 18/30 : RUN echo "Downloading Miniconda installer ..." && miniconda_installer=/tmp/miniconda.sh && curl -sSL --retry 5 -o $miniconda_installer https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh && /bin/bash $miniconda_installer -b -p $CONDA_DIR && rm -f $miniconda_installer && conda config --system --prepend channels conda-forge && conda config --system --set auto_update_conda false && conda config --system --set show_channel_urls true && conda clean -tipsy && sync
---> Using cache
---> 151b9294db01
Step 19/30 : RUN conda create -y -q --name neuro python=3.6 jupyter jupyterlab jupyter_contrib_nbextensions traits pandas matplotlib scikit-learn seaborn && sync && conda clean -tipsy && sync && /bin/bash -c "source activate neuro && pip install -q --no-cache-dir https://github.com/nipy/nipype/tarball/master https://github.com/INCF/pybids/tarball/master nilearn datalad[full] nipy duecredit" && sync && sed -i '$isource activate neuro' $ND_ENTRYPOINT
---> Running in 9da038ddc954
Solving environment: ...working... done
## Package Plan ##
environment location: /opt/conda/envs/neuro
added / updated specs:
- jupyter
- jupyter_contrib_nbextensions
- jupyterlab
- matplotlib
- pandas
- python=3.6
- scikit-learn
- seaborn
- traits
The following packages will be downloaded:
package | build
---------------------------|-----------------
readline-7.0 | 0 838 KB conda-forge
testpath-0.3.1 | py36_0 90 KB conda-forge
seaborn-0.8.1 | py36_0 334 KB conda-forge
pcre-8.39 | 0 725 KB conda-forge
jupyter_contrib_core-0.3.3 | py36_1 28 KB conda-forge
scikit-learn-0.19.1 |py36_blas_openblas_201 12.8 MB conda-forge
ipykernel-4.8.2 | py36_0 148 KB conda-forge
pyzmq-17.0.0 | py36_3 1.1 MB conda-forge
numpy-1.14.1 |py36_blas_openblas_200 9.0 MB conda-forge
jupyter_client-5.2.2 | py36_0 126 KB conda-forge
pyparsing-2.2.0 | py36_0 95 KB conda-forge
pygments-2.2.0 | py36_0 1.3 MB conda-forge
qtconsole-4.3.1 | py36_0 149 KB conda-forge
zeromq-4.2.3 | 2 881 KB conda-forge
simplegeneric-0.8.1 | py36_0 8 KB conda-forge
libsodium-1.0.15 | 1 1.5 MB conda-forge
python-dateutil-2.6.1 | py36_0 237 KB conda-forge
setuptools-38.5.1 | py36_0 524 KB conda-forge
freetype-2.8.1 | 0 2.9 MB conda-forge
icu-58.2 | 0 22.7 MB conda-forge
scipy-1.0.0 |py36_blas_openblas_201 40.3 MB conda-forge
libpng-1.6.34 | 0 305 KB conda-forge
gst-plugins-base-1.8.0 | 0 4.0 MB conda-forge
jupyter_core-4.4.0 | py_0 44 KB conda-forge
jupyter_console-5.2.0 | py36_0 34 KB conda-forge
mistune-0.8.3 | py_0 16 KB conda-forge
jupyter_highlight_selected_word-0.1.0| py36_0 13 KB conda-forge
entrypoints-0.2.3 | py36_1 9 KB conda-forge
jupyter-1.0.0 | py36_0 3 KB conda-forge
pexpect-4.4.0 | py36_0 74 KB conda-forge
blas-1.1 | openblas 1 KB conda-forge
ncurses-5.9 | 10 1.1 MB conda-forge
lxml-4.1.1 | py36_0 5.4 MB conda-forge
wcwidth-0.1.7 | py36_0 23 KB conda-forge
jupyterlab-0.31.8 | py36_1 8.5 MB conda-forge
backports.functools_lru_cache-1.5| py36_0 11 KB conda-forge
gmp-6.1.2 | 0 670 KB conda-forge
sqlite-3.20.1 | 2 1.3 MB conda-forge
webencodings-0.5 | py36_0 18 KB conda-forge
yaml-0.1.6 | 0 300 KB conda-forge
jsonschema-2.6.0 | py36_1 64 KB conda-forge
jupyter_latex_envs-1.4.0 | py36_1 738 KB conda-forge
xorg-libxau-1.0.8 | 3 14 KB conda-forge
gettext-0.19.8.1 | 0 6.3 MB conda-forge
jinja2-2.10 | py36_0 183 KB conda-forge
prompt_toolkit-1.0.15 | py36_0 339 KB conda-forge
ipython_genutils-0.2.0 | py36_0 36 KB conda-forge
notebook-5.4.0 | py36_0 6.6 MB conda-forge
ipywidgets-7.1.2 | py36_0 99 KB conda-forge
tk-8.6.7 | 0 3.1 MB conda-forge
openssl-1.0.2n | 0 3.5 MB conda-forge
xorg-libxdmcp-1.1.2 | 3 21 KB conda-forge
sip-4.18 | py36_1 263 KB conda-forge
libxslt-1.1.32 | 0 1.4 MB conda-forge
pyyaml-3.12 | py36_1 394 KB conda-forge
libiconv-1.15 | 0 2.0 MB conda-forge
backports-1.0 | py36_1 4 KB conda-forge
statsmodels-0.8.0 | py36_0 7.9 MB conda-forge
html5lib-1.0.1 | py_0 89 KB conda-forge
zlib-1.2.11 | 0 93 KB conda-forge
ipython-6.2.1 | py36_1 1.0 MB conda-forge
pandas-0.22.0 | py36_0 26.1 MB conda-forge
jpeg-9b | 2 1.0 MB conda-forge
fontconfig-2.12.6 | 0 943 KB conda-forge
jupyter_nbextensions_configurator-0.4.0| py36_0 470 KB conda-forge
cycler-0.10.0 | py36_0 14 KB conda-forge
nbformat-4.4.0 | py36_0 140 KB conda-forge
markupsafe-1.0 | py36_0 32 KB conda-forge
openblas-0.2.20 | 7 17.0 MB conda-forge
libffi-3.2.1 | 3 47 KB conda-forge
patsy-0.5.0 | py36_0 322 KB conda-forge
dbus-1.10.22 | 0 1.6 MB conda-forge
pandocfilters-1.4.1 | py36_0 13 KB conda-forge
tornado-4.5.3 | py36_0 628 KB conda-forge
traitlets-4.3.2 | py36_0 131 KB conda-forge
traits-4.6.0 | py36_1 468 KB conda-forge
bleach-2.0.0 | py_1 20 KB conda-forge
xz-5.2.3 | 0 854 KB conda-forge
libxml2-2.9.7 | 0 5.0 MB conda-forge
libgfortran-3.0.0 | 1 281 KB defaults
python-3.6.4 | 0 19.1 MB conda-forge
jupyterlab_launcher-0.10.5 | py36_0 24 KB conda-forge
expat-2.2.5 | 0 434 KB conda-forge
jedi-0.11.1 | py36_0 306 KB conda-forge
ca-certificates-2018.1.18 | 0 140 KB conda-forge
pickleshare-0.7.4 | py36_0 11 KB conda-forge
send2trash-1.5.0 | py_0 12 KB conda-forge
pytz-2018.3 | py_0 217 KB conda-forge
matplotlib-2.1.2 | py36_0 9.4 MB conda-forge
terminado-0.8.1 | py36_0 20 KB conda-forge
ptyprocess-0.5.2 | py36_0 22 KB conda-forge
six-1.11.0 | py36_1 21 KB conda-forge
gstreamer-1.8.0 | 1 3.3 MB conda-forge
glib-2.55.0 | 0 8.3 MB conda-forge
widgetsnbextension-3.1.4 | py36_0 1.7 MB conda-forge
pyqt-5.6.0 | py36_4 5.3 MB conda-forge
nbconvert-5.3.1 | py_1 320 KB conda-forge
qt-5.6.2 | 7 44.3 MB conda-forge
wheel-0.30.0 | py36_2 66 KB conda-forge
pip-9.0.1 | py36_1 1.7 MB conda-forge
pandoc-2.1.1 | 0 20.8 MB conda-forge
certifi-2018.1.18 | py36_0 143 KB conda-forge
jupyter_contrib_nbextensions-0.4.0| py36_0 19.4 MB conda-forge
parso-0.1.1 | py_0 60 KB conda-forge
decorator-4.1.2 | py36_0 15 KB conda-forge
libxcb-1.12 | 1 454 KB conda-forge
------------------------------------------------------------
Total: 342.4 MB
The following NEW packages will be INSTALLED:
backports: 1.0-py36_1 conda-forge
backports.functools_lru_cache: 1.5-py36_0 conda-forge
blas: 1.1-openblas conda-forge
bleach: 2.0.0-py_1 conda-forge
ca-certificates: 2018.1.18-0 conda-forge
certifi: 2018.1.18-py36_0 conda-forge
cycler: 0.10.0-py36_0 conda-forge
dbus: 1.10.22-0 conda-forge
decorator: 4.1.2-py36_0 conda-forge
entrypoints: 0.2.3-py36_1 conda-forge
expat: 2.2.5-0 conda-forge
fontconfig: 2.12.6-0 conda-forge
freetype: 2.8.1-0 conda-forge
gettext: 0.19.8.1-0 conda-forge
glib: 2.55.0-0 conda-forge
gmp: 6.1.2-0 conda-forge
gst-plugins-base: 1.8.0-0 conda-forge
gstreamer: 1.8.0-1 conda-forge
html5lib: 1.0.1-py_0 conda-forge
icu: 58.2-0 conda-forge
ipykernel: 4.8.2-py36_0 conda-forge
ipython: 6.2.1-py36_1 conda-forge
ipython_genutils: 0.2.0-py36_0 conda-forge
ipywidgets: 7.1.2-py36_0 conda-forge
jedi: 0.11.1-py36_0 conda-forge
jinja2: 2.10-py36_0 conda-forge
jpeg: 9b-2 conda-forge
jsonschema: 2.6.0-py36_1 conda-forge
jupyter: 1.0.0-py36_0 conda-forge
jupyter_client: 5.2.2-py36_0 conda-forge
jupyter_console: 5.2.0-py36_0 conda-forge
jupyter_contrib_core: 0.3.3-py36_1 conda-forge
jupyter_contrib_nbextensions: 0.4.0-py36_0 conda-forge
jupyter_core: 4.4.0-py_0 conda-forge
jupyter_highlight_selected_word: 0.1.0-py36_0 conda-forge
jupyter_latex_envs: 1.4.0-py36_1 conda-forge
jupyter_nbextensions_configurator: 0.4.0-py36_0 conda-forge
jupyterlab: 0.31.8-py36_1 conda-forge
jupyterlab_launcher: 0.10.5-py36_0 conda-forge
libffi: 3.2.1-3 conda-forge
libgfortran: 3.0.0-1 defaults
libiconv: 1.15-0 conda-forge
libpng: 1.6.34-0 conda-forge
libsodium: 1.0.15-1 conda-forge
libxcb: 1.12-1 conda-forge
libxml2: 2.9.7-0 conda-forge
libxslt: 1.1.32-0 conda-forge
lxml: 4.1.1-py36_0 conda-forge
markupsafe: 1.0-py36_0 conda-forge
matplotlib: 2.1.2-py36_0 conda-forge
mistune: 0.8.3-py_0 conda-forge
nbconvert: 5.3.1-py_1 conda-forge
nbformat: 4.4.0-py36_0 conda-forge
ncurses: 5.9-10 conda-forge
notebook: 5.4.0-py36_0 conda-forge
numpy: 1.14.1-py36_blas_openblas_200 conda-forge [blas_openblas]
openblas: 0.2.20-7 conda-forge
openssl: 1.0.2n-0 conda-forge
pandas: 0.22.0-py36_0 conda-forge
pandoc: 2.1.1-0 conda-forge
pandocfilters: 1.4.1-py36_0 conda-forge
parso: 0.1.1-py_0 conda-forge
patsy: 0.5.0-py36_0 conda-forge
pcre: 8.39-0 conda-forge
pexpect: 4.4.0-py36_0 conda-forge
pickleshare: 0.7.4-py36_0 conda-forge
pip: 9.0.1-py36_1 conda-forge
prompt_toolkit: 1.0.15-py36_0 conda-forge
ptyprocess: 0.5.2-py36_0 conda-forge
pygments: 2.2.0-py36_0 conda-forge
pyparsing: 2.2.0-py36_0 conda-forge
pyqt: 5.6.0-py36_4 conda-forge
python: 3.6.4-0 conda-forge
python-dateutil: 2.6.1-py36_0 conda-forge
pytz: 2018.3-py_0 conda-forge
pyyaml: 3.12-py36_1 conda-forge
pyzmq: 17.0.0-py36_3 conda-forge
qt: 5.6.2-7 conda-forge
qtconsole: 4.3.1-py36_0 conda-forge
readline: 7.0-0 conda-forge
scikit-learn: 0.19.1-py36_blas_openblas_201 conda-forge [blas_openblas]
scipy: 1.0.0-py36_blas_openblas_201 conda-forge [blas_openblas]
seaborn: 0.8.1-py36_0 conda-forge
send2trash: 1.5.0-py_0 conda-forge
setuptools: 38.5.1-py36_0 conda-forge
simplegeneric: 0.8.1-py36_0 conda-forge
sip: 4.18-py36_1 conda-forge
six: 1.11.0-py36_1 conda-forge
sqlite: 3.20.1-2 conda-forge
statsmodels: 0.8.0-py36_0 conda-forge
terminado: 0.8.1-py36_0 conda-forge
testpath: 0.3.1-py36_0 conda-forge
tk: 8.6.7-0 conda-forge
IsADirectoryError(21, 'Is a directory')
tornado: 4.5.3-py36_0 conda-forge
traitlets: 4.3.2-py36_0 conda-forge
traits: 4.6.0-py36_1 conda-forge
wcwidth: 0.1.7-py36_0 conda-forge
webencodings: 0.5-py36_0 conda-forge
wheel: 0.30.0-py36_2 conda-forge
widgetsnbextension: 3.1.4-py36_0 conda-forge
xorg-libxau: 1.0.8-3 conda-forge
xorg-libxdmcp: 1.1.2-3 conda-forge
xz: 5.2.3-0 conda-forge
yaml: 0.1.6-0 conda-forge
zeromq: 4.2.3-2 conda-forge
zlib: 1.2.11-0 conda-forge
The command '/bin/sh -c conda create -y -q --name neuro python=3.6 jupyter jupyterlab jupyter_contrib_nbextensions traits pandas matplotlib scikit-learn seaborn && sync && conda clean -tipsy && sync && /bin/bash -c "source activate neuro && pip install -q --no-cache-dir https://github.com/nipy/nipype/tarball/master https://github.com/INCF/pybids/tarball/master nilearn datalad[full] nipy duecredit" && sync && sed -i '$isource activate neuro' $ND_ENTRYPOINT' returned a non-zero code: 1