Hi,
I just spent a week tearing out my hair, trying to get a bare metal fmriprep141 going (yes, I know I should use a container, but the two large clusters I have access to are still “evaluating” singularity after over a year, and docker is a complete non-starter). I FINALLY got it to work today, and thought I’d share my experience to save others from it…
There seem to be a number of issues having to do with the recent move to TemplateFlow - almost all of my failures were around fetching templates. For example:
/PHShome/bbf2/software/anaconda3/envs/fmriprep141sing/lib/python3.7/site-packages/fmriprep/cli/run.py:492: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
plugin_settings = loadyml(f)
Process Process-2:
Traceback (most recent call last):
File "/PHShome/bbf2/software/anaconda3/envs/fmriprep141sing/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/PHShome/bbf2/software/anaconda3/envs/fmriprep141sing/lib/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "/PHShome/bbf2/software/anaconda3/envs/fmriprep141sing/lib/python3.7/site-packages/fmriprep/cli/run.py", line 610, in build_workflow
work_dir=str(work_dir),
File "/PHShome/bbf2/software/anaconda3/envs/fmriprep141sing/lib/python3.7/site-packages/fmriprep/workflows/base.py", line 259, in init_fmriprep_wf
use_syn=use_syn,
File "/PHShome/bbf2/software/anaconda3/envs/fmriprep141sing/lib/python3.7/site-packages/fmriprep/workflows/base.py", line 551, in init_single_subject_wf
skull_strip_template=skull_strip_template,
File "/PHShome/bbf2/software/anaconda3/envs/fmriprep141sing/lib/python3.7/site-packages/smriprep/workflows/anatomical.py", line 230, in init_anat_preproc_wf
normalization_quality='precise' if not debug else 'testing')
File "/PHShome/bbf2/software/anaconda3/envs/fmriprep141sing/lib/python3.7/site-packages/niworkflows/anat/ants.py", line 183, in init_brain_extraction_wf
template_spec=template_spec)
File "/PHShome/bbf2/software/anaconda3/envs/fmriprep141sing/lib/python3.7/site-packages/niworkflows/utils/misc.py", line 54, in get_template_specs
argument.""".format(in_template, template_spec))
RuntimeError: Could not find template "OASIS30ANTs" with specs={'suffix': 'T1w', 'desc': None, 'atlas': None, 'resolution': 1}. Please revise your template argument.(by the way, I’m not using OASIS30ANTs as one of my output spaces).
Since I did install 1.4.1 in a singularity container on our local (small) cluster, and running the same subject from the same dataset worked fine - no template related errors (one functional processing error, but that’s probably the data itself), I figured it was my setup, not my data or my command line options. Also, none of the external non-python dependencies had changed, and they worked for me for fmriprep 1.2.6 and 1.3.2, so I figured it wasn’t that.
So I replicated the containerized environment as closely as possible.
I started a shell in the singularity container and cloned the python environment using:
conda env export > fmriprep141.ymlI copied that file onto the big cluster, and edited the first line to change the name from “base” to fmriprep141, then created a duplicate environment on my big cluster with:
conda env create -f fmriprep141.ymlActivated the new environment:
source activate fmriprep141Running fmriprep then failed because git-annex didn’t exist. Fixed that with:
conda install git-annexThen I proceeded to the the stage where the template errors occurred. I tried it with and without setting the TEMPLATEFLOW_HOME environment variable and it failed every time. Turns out it seems to be some sort of conflict between TemplateFlow versions.
I found an existing set of templates in my ${HOME}/.cache directory. I don’t know what version of TemplateFlow they were coming from, but they were NOT compatible with what fmriprep 1.4.1 wanted. Blew away ${HOME}/.cache/templateflow, reran, and all was well.
Well, almost all was well. I had a work directory around from analyzing that data with a previous version of fmriprep, which caused some other crash because the analysis structure seemed to have changed between versions. Wiped that directory clean, and NOW all was well!
Clearly, containers are the way to go, but if that’s not an option, bare metal installs do work…
I’m including the contents of fmriprep141.yml below:
name: fmriprep141
channels:
- defaults
dependencies:
- asn1crypto=0.24.0=py37_0
- blas=1.0=mkl
- ca-certificates=2019.1.23=0
- cairo=1.14.12=h8948797_3
- certifi=2019.3.9=py37_0
- cffi=1.11.5=py37he75722e_1
- chardet=3.0.4=py37_1
- conda=4.6.14=py37_0
- conda-env=2.6.0=1
- cryptography=2.6.1=py37h1ba5d50_0
- cycler=0.10.0=py37_0
- dbus=1.13.6=h746ee38_0
- expat=2.2.6=he6710b0_0
- fontconfig=2.13.0=h9420a91_0
- freetype=2.9.1=h8a8886c_1
- fribidi=1.0.5=h7b6447c_0
- glib=2.56.2=hd408876_0
- graphite2=1.3.13=h23475e2_0
- graphviz=2.40.1=h21bd128_2
- gst-plugins-base=1.14.0=hbbd80ab_1
- gstreamer=1.14.0=hb453b48_1
- harfbuzz=1.8.8=hffaf4a1_0
- icu=58.2=h9c2bf20_1
- idna=2.7=py37_0
- intel-openmp=2019.3=199
- jpeg=9b=h024ee3a_2
- kiwisolver=1.1.0=py37he6710b0_0
- libedit=3.1.20170329=h6b74fdf_2
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=8.2.0=hdf63c60_1
- libgfortran-ng=7.3.0=hdf63c60_0
- libpng=1.6.37=hbc83047_0
- libstdcxx-ng=8.2.0=hdf63c60_1
- libtiff=4.0.10=h2733197_2
- libuuid=1.0.3=h1bed415_2
- libxcb=1.13=h1bed415_1
- libxml2=2.9.8=h26e45fe_1
- libxslt=1.1.32=h1312cb7_0
- matplotlib=2.2.2=py37hb69df0a_2
- mkl=2018.0.3=1
- mkl-service=1.1.2=py37h90e4bf4_5
- mkl_fft=1.0.6=py37h7dd41cf_0
- mkl_random=1.0.1=py37h4414c95_1
- ncurses=6.1=hf484d3e_0
- numpy=1.15.4=py37h1d66e8a_0
- numpy-base=1.15.4=py37h81de0dd_0
- openssl=1.1.1b=h7b6447c_1
- pandas=0.23.4=py37h04863e7_0
- pango=1.42.4=h049681c_0
- pcre=8.43=he6710b0_0
- pip=19.1=py37_0
- pixman=0.38.0=h7b6447c_0
- pycosat=0.6.3=py37h14c3975_0
- pycparser=2.18=py37_1
- pyopenssl=18.0.0=py37_0
- pyparsing=2.4.0=py_0
- pyqt=5.9.2=py37h05f1152_2
- pysocks=1.6.8=py37_0
- python=3.7.1=h0371630_7
- python-dateutil=2.8.0=py37_0
- pytz=2019.1=py_0
- qt=5.9.7=h5867ecd_1
- readline=7.0=h7b6447c_5
- requests=2.19.1=py37_0
- ruamel_yaml=0.15.46=py37h14c3975_0
- scikit-learn=0.19.1=py37hedc7406_0
- scipy=1.1.0=py37hfa4b5c9_1
- setuptools=40.2.0=py37_0
- sip=4.19.8=py37hf484d3e_0
- six=1.11.0=py37_1
- sqlite=3.26.0=h7b6447c_0
- tk=8.6.8=hbc83047_0
- tornado=6.0.2=py37h7b6447c_0
- traits=4.6.0=py37h7b6447c_3
- urllib3=1.23=py37_0
- wheel=0.31.1=py37_0
- xz=5.2.4=h14c3975_4
- yaml=0.1.7=had09818_2
- zlib=1.2.11=h7b6447c_3
- zstd=1.3.7=h0b5b093_0
- pip:
- alabaster==0.7.12
- appdirs==1.4.3
- atomicwrites==1.3.0
- attrs==19.1.0
- babel==2.7.0
- bids-validator==1.2.4
- bleach==3.1.0
- boto==2.49.0
- citeproc-py==0.4.0
- click==7.0
- codecov==2.0.15
- coverage==4.5.3
- cython==0.29.12
- datalad==0.11.5
- decorator==4.4.0
- defusedxml==0.6.0
- deprecated==1.2.6
- docopt==0.6.2
- docutils==0.14
- duecredit==0.7.0
- entrypoints==0.3
- fasteners==0.15
- fmriprep==1.4.1
- funcsigs==1.0.2
- future==0.17.1
- gitdb2==2.0.5
- gitpython==2.1.11
- grabbit==0.2.6
- humanize==0.5.1
- imageio==2.5.0
- imagesize==1.1.0
- importlib-metadata==0.18
- indexed-gzip==0.8.10
- ipython-genutils==0.2.0
- iso8601==0.1.12
- isodate==0.6.0
- jeepney==0.4
- jinja2==2.10.1
- jsmin==2.2.2
- jsonschema==3.0.1
- jupyter-core==4.5.0
- keyring==19.0.2
- keyrings-alt==3.1.1
- lockfile==0.12.2
- lxml==4.3.4
- markupsafe==1.1.1
- mistune==0.8.4
- mock==3.0.5
- monotonic==1.5
- more-itertools==7.1.0
- msgpack==0.6.1
- nbconvert==5.5.0
- nbformat==4.4.0
- nbsphinx==0.4.2
- networkx==2.3
- neurdflib==5.0.0.post1
- nibabel==2.4.1
- nilearn==0.5.2
- nipype==1.2.0
- nitime==0.8.1
- niworkflows==0.9.6
- num2words==0.5.10
- packaging==19.0
- pandocfilters==1.4.2
- patool==1.12
- patsy==0.5.1
- pillow==6.1.0
- pluggy==0.12.0
- prov==1.5.3
- psutil==5.6.3
- py==1.8.0
- pybids==0.7.1
- pydot==1.4.1
- pydotplus==2.0.2
- pygithub==1.43.7
- pygments==2.4.2
- pyjwt==1.7.1
- pyrsistent==0.15.3
- pytest==5.0.1
- pywavelets==1.0.3
- pyyaml==5.1.1
- rdflib==4.2.2
- scikit-image==0.15.0
- seaborn==0.9.0
- secretstorage==3.1.1
- sentry-sdk==0.10.0
- simplejson==3.16.0
- smmap2==2.0.5
- smriprep==0.2.4
- snowballstemmer==1.9.0
- sphinx==2.1.2
- sphinx-argparse==0.2.5
- sphinx-rtd-theme==0.4.3
- sphinxcontrib-applehelp==1.0.1
- sphinxcontrib-devhelp==1.0.1
- sphinxcontrib-htmlhelp==1.0.2
- sphinxcontrib-jsmath==1.0.1
- sphinxcontrib-qthelp==1.0.2
- sphinxcontrib-serializinghtml==1.1.3
- statsmodels==0.10.0
- svgutils==0.3.1
- tedana==0.0.7
- templateflow==0.3.0
- testpath==0.4.2
- tqdm==4.32.1
- traitlets==4.3.2
- versioneer==0.18
- wcwidth==0.1.7
- webencodings==0.5.1
- whoosh==2.7.4
- wrapt==1.11.2
- zipp==0.5.2
prefix: /usr/local/miniconda
Blaise