Connection error with fmriprep (using singularity and SLURM on an offline HPC)

Hi everyone!
I’m using fmriprep version 20-1-1 via Singularity on an offline HPC. I have been getting this error, although I added the --notrack comment and have manually set an environment for my templateflow folder that I transferred to the server.

Downloading https://templateflow.s3.amazonaws.com/tpl-OASIS30ANTs/tpl-OASIS30ANTs_res-01_T1w.nii.gz
Process Process-2:
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/site-packages/urllib3/connection.py", line 171, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw)
  File "/usr/local/miniconda/lib/python3.7/site-packages/urllib3/util/connection.py", line 56, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/usr/local/miniconda/lib/python3.7/socket.py", line 748, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/local/miniconda/lib/python3.7/site-packages/urllib3/connectionpool.py", line 343, in _make_request
    self._validate_conn(conn)
  File "/usr/local/miniconda/lib/python3.7/site-packages/urllib3/connectionpool.py", line 849, in _validate_conn
    conn.connect()
  File "/usr/local/miniconda/lib/python3.7/site-packages/urllib3/connection.py", line 314, in connect
    conn = self._new_conn()
  File "/usr/local/miniconda/lib/python3.7/site-packages/urllib3/connection.py", line 180, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x2ade782b5a20>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/site-packages/requests/adapters.py", line 445, in send
    timeout=timeout
  File "/usr/local/miniconda/lib/python3.7/site-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/local/miniconda/lib/python3.7/site-packages/urllib3/util/retry.py", line 398, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='templateflow.s3.amazonaws.com', port=443): Max retries exceeded with url: /tpl-OASIS30ANTs/tpl-OASIS30ANTs_res-01_T1w.nii.gz (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x2ade782b5a20>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/cli/workflow.py", line 84, in build_workflow
    retval["workflow"] = init_fmriprep_wf()
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/workflows/base.py", line 64, in init_fmriprep_wf
    single_subject_wf = init_single_subject_wf(subject_id)
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/workflows/base.py", line 251, in init_single_subject_wf
    t1w=subject_data['t1w'],
  File "/usr/local/miniconda/lib/python3.7/site-packages/smriprep/workflows/anatomical.py", line 317, in init_anat_preproc_wf
    normalization_quality='precise' if not debug else 'testing')
  File "/usr/local/miniconda/lib/python3.7/site-packages/niworkflows/anat/ants.py", line 167, in init_brain_extraction_wf
    in_template, template_spec=template_spec
  File "/usr/local/miniconda/lib/python3.7/site-packages/niworkflows/utils/misc.py", line 60, in get_template_specs
    tpl_target_path = get_template(in_template, **template_spec)
  File "/usr/local/miniconda/lib/python3.7/site-packages/templateflow/api.py", line 68, in get
    _s3_get(filepath)
  File "/usr/local/miniconda/lib/python3.7/site-packages/templateflow/api.py", line 208, in _s3_get
    r = requests.get(url, stream=True)
  File "/usr/local/miniconda/lib/python3.7/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/requests/sessions.py", line 512, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/requests/sessions.py", line 622, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/requests/adapters.py", line 513, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='templateflow.s3.amazonaws.com', port=443): Max retries exceeded with url: /tpl-OASIS30ANTs/tpl-OASIS30ANTs_res-01_T1w.nii.gz (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x2ade782b5a20>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))

Some extra info:

Here is my slrum command:
#!/bin/bash -l
#SBATCH -A sens2020508
#SBATCH -J fmriprep # job name
#SBATCH --array=1-100%5
#SBATCH --time=16:00:00 # Wallclock time
#SBATCH -n 1
#SBATCH --cpus-per-task=16
#SBATCH --exclusive

# Outputs ----------------------------------
#SBATCH -o %A-%a.out
#SBATCH -e %A-%a.err
#SBATCH --mail-user=irem.undeger@ki.se
#SBATCH --mail-type=ALL
# ------------------------------------------
unset PYTHONPATH
export SINGULARITYENV_TEMPLATEFLOW_HOME=/castor/project/proj/fmriprep/.cache/templateflow

eval $(sed "${SLURM_ARRAY_TASK_ID}q;d" task1.sh )

Here is “task1.sh”:

singularity run -B $HOME/.cache:/home/fmriprep/.cache --cleanenv /castor/project/proj/singularity/fmriprep_20-1-1.simg --participant-label 35 --nthreads 5 --omp-nthreads 10 --ignore slicetiming --output-spaces MNI152NLin6Asym:res-2 MNI152NLin6Asym anat --fs-license-file /castor/project/proj/license.txt -w /castor/project/proj/work bids3 castor/project/proj/bids3/derivatives participant --notrack

Note: I have tried without “-B $HOME/.cache:/home/fmriprep/.cache” and got the same error. In that case I had the template flow folder outside of the cache and had set the environment accordingly.

Thank you for all your help in advance!