Singularity fmriprep "export.opts" error

singularity
freesurfer
fmriprep

#1

Hi all, I’m trying to use singularity (2.4.4) to run fmriprep (1.0.8) on one example subject on a Linux server. I’m having this persistent problem where, when it gets to the FreeSurfer stage, it tries to write the ‘/expert.opts’ config file and hits a read-only error. (The bids-validator passes and BIDS Apps like MRIQC work fine.) I’m relatively new to singularity, so maybe this is just a dumb problem with how I’m binding directories…

I start by building the singularity image from docker:
singularity build fmriprep-v1.0.8.sqsh docker://poldracklab/fmriprep
(but I get the same error if I use docker2singularity…)

Then I invoke singularity to run fmriprep with the following:

singularity run --cleanenv \
    --bind /usr/people/snastase:/home \
    /usr/people/snastase/singularity/fmriprep-v1.0.8-new.sqsh \
    --participant-label sub-01 \
    --fs-license-file /home/storyteller/data/code/license.txt \
    --bold2t1w-dof 9 --nthreads 4 --omp-nthreads 4 \
    --output-space T1w template fsaverage6 \
    --template MNI152NLin2009cAsym \
    --use-syn-sdc --write-graph /home/storyteller/data \
    -w /home/storyteller/data/derivatives/work \
    /home/storyteller/data/derivatives participant

This works through several steps, but then breaks at the “Creating BOLD surface-sampling workflow” stage, with the following error:

180327-23:00:19,449 workflow WARNING:
     [Node] Error on "fmriprep_wf.single_subject_01_wf.anat_preproc_wf.surface_recon_wf.autorecon1" (/home/storyteller/data/derivatives/work/fmriprep_wf/single_subject_01_wf/anat_preproc_wf/surface_recon_wf/autorecon1)
180327-23:00:20,659 workflow ERROR:
     Node autorecon1 failed to run on host spock-login.pni.princeton.edu.
180327-23:00:20,663 workflow ERROR:
     Saving crash info to /home/storyteller/data/derivatives/fmriprep/sub-01/log/20180327-225912_57f6423a-8da0-4b6e-8785-bb2d97d95c99/crash-20180327-230020-snastase-autorecon1-4932f6e1-5015-4e08-aa16-89053a0906a4.txt
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/plugins/multiproc.py", line 68, in run_node
    result['result'] = node.run(updatehash=updatehash)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/nodes.py", line 487, in run 
    result = self._run_interface(execute=True)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/nodes.py", line 571, in _run_interface
    return self._run_command(execute)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/nodes.py", line 638, in _run_command
    cmd = self._interface.cmdline
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/interfaces/freesurfer/preprocess.py", line 1235, in cmdline
    cmd += self._prep_expert_file()
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/interfaces/freesurfer/preprocess.py", line 1318, in _prep_expert_file
    with open(expert_fname, 'w') as fobj:
OSError: [Errno 30] Read-only file system: '/expert.opts'

180327-23:00:36,667 workflow ERROR:
     could not run node: fmriprep_wf.single_subject_01_wf.anat_preproc_wf.surface_recon_wf.autorecon1
180327-23:00:36,741 cli WARNING:
     Errors occurred while generating reports for participants: 01 (1).

Let me know if you have any ideas where I’m going wrong. Thanks!


#2

Interesting… Is this a submm dataset? Turning off the submm recon all might work around this issue --no-submm-recon

Other thing I can think of is interference of your host environment variables. Passing the --cleanenv argument to singularity (not FMRIPREP) might test this theory.


#3

Oh, good call. Yes, the structural images are 0.90 x 0.86 x 0.86. If I use --no-submm-recon, it seems to bypass this error and is now chugging along. I assume this won’t radically alter the quality / alignment of the surface reconstruction, but will have a look at the output. Thanks!


#4

I opened an issue for the underlying problem: https://github.com/nipy/nipype/issues/2518


#5

hey @snastase what is your working directory in the host machine?


#6

@snastase Are you able to share a defaced copy of the T1w image? If we can reliably produce the error, that should help us track down exactly how we’re ending up in the wrong working directory.


#7

@oesteban I’m running the singularity command from /usr/people/snastase/storyteller/data/code, where data/ is the top-level BIDS directory, but maybe I should call it from one level up.

@effigies Will get back to you with a defaced anatomical shortly…


#8

Okay, I figured out the issue. It looks like your commands are being run from a working directory of /. If you can convince them to run from a directory in which you have write access, then you should at least be able to correctly run fmriprep with sub-millimeter reconstruction. You will be getting an expert.opts file in that directory, as well, which you may safely delete.

The next version of fmriprep should resolve this.