Fmriprep: ERRO[7120] error getting events from daemon: EOF

Hi there,

I have just started to try out fmriprep and seem to be running into problems very early in the process.

I have a locally collected dataset that I’ve set up in BIDS format (verified) as well as a dataset (ds008) that I downloaded from the BIDS website.

I’m running fmriprep (downloaded yesterday) with Docker 17.03.1-ce-mac12 on Mac OS X Yosemite 10.10.5.

Regardless of how I run fmriprep or the dataset I run it on, I seem to get the same error:
ERRO[7120] error getting events from daemon: EOF

I started with trying to run fmriprep with all the default settings, and the error seemed to appear when the call to ANTs was made. There were no outputs.

I then ran:
docker run -ti --rm -v ~/fmri/anderson_ADHD/BIDS/ds008:/data:ro -v ~/fmri/anderson_ADHD/BIDS/ds008/out:/out poldracklab/fmriprep:latest --participant_label 03 --no-skull-strip-ants --no-freesurfer /data /out/out participant
In the out/sub/anat directory, the _brainmask, _dtissue and _preproc files have been produced. In the out/sub/func directory, there are _confounds.tsv files for each run, but that’s it.
There are no logs or other outputs.

I ran the same command with my local dataset and obtained the same results.

Here are the last few lines:
170525-11:10:53,775 workflow INFO:
[Job finished] jobname: concat_rois jobid: 237
170525-11:10:53,796 workflow INFO:
Executing: signals ID: 238
170525-11:10:53,801 workflow INFO:
Executing node signals in dir: /root/src/fmriprep/work/fmriprep_wf/single_subject_S19_wf/func_preproc_task_rest_wf/discover_wf/signals
ERRO[7120] error getting events from daemon: EOF

Any ideas? Sorry if I am getting something fundamentally wrong - docker is very new to me!

Clare

1 Like

Hi @immalab, thanks for the report!

It looks like this issue may be related to memory and/or number of CPUs allocated to Docker. We recommend running FMRIPREP with at least 8GB of memory. We don’t have a minimum CPU requirement, so that’s unlikely to be the issue.


Just as something else to consider: We now recommend using the fmriprep-docker command-line tool to run the Docker images. This should warn you about memory usage, as well as reduce the potential for misconfigured Docker options. For instance, your above command would become:

fmriprep-docker --participant_label 03 --no-skull-strip-ants --no-freesurfer \
    ~/fmrip/anderson_ADHD/BIDS/ds008 ~/fmri/anderson_ADHD/BIDS/ds008/out/out \
    participant

You can install it with

pip install fmriprep-docker

Thank you @effigies. I increased the RAM and number of CPUs to the max possible, and was able to get a lot further in the process. I ran the command-line tool as recommended, though I seem to be having some issues with the path.

Will keep at it… thanks!
Clare

A follow-up question about fmriprep is whether when you rerun it for a dataset, does it skip over steps already completed or redo everything? Can you set a option or overwrite or not?

Sorry if this is very basic info, I don’t see it in the readthedocs.

Thanks,
Clare

Hi Clare,

By default, FMRIPREP will reuse previously run steps, but only if the working directory is persistent. For fmriprep-docker, use the -w <working_directory> option to specify a working directory, otherwise a temporary one will be created inside the docker image, which will be deleted upon termination.

To modify the above example to use a persistent directory called scratch:

fmriprep-docker --participant_label 03 --no-skull-strip-ants --no-freesurfer \
    ~/fmrip/anderson_ADHD/BIDS/ds008 ~/fmri/anderson_ADHD/BIDS/ds008/out/out \
    participant -w ~/fmri/anderson_ADHD/scratch

There is no direct option to overwrite; you can remove the scratch directory (or specify a different one) to avoid reusing previous results.

We should update the documentation to make this clear.

1 Like

Thanks @effigies.

Will the use of the scratch directory help with troubleshooting? At the moment, things are still only running part of the way through. I am getting some outputs, for some subjects, but no reports or log files.

I keep getting that same error, even though I’ve increased the RAM and CPUs in Docker to the max. I’m going to move to working on our HPC cluster but the staff there were not confident that the singularity container was going to work well.

Thanks,
Clare

Hi Clare,

The scratch directory can definitely be useful for inspecting intermediate results. You can also create the following nipype.cfg:

[logging]
log_directory = /scratch
log_to_file = true

[execution]
stop_on_first_crash = true

You can then add --config nipype.cfg to your fmriprep-docker command, to generate a run log in your scratch directory. The stop_on_first_crash will keep the result from getting buried. If it’s still hard to find the failing interface, you can use --nthreads 1 to ensure that the failing interface will be the last one to run.

Feel free to post the log as a gist if you want us to have a look.

Hi Clare,
Have you tried reinstalling or updating docker on your computer?

As for Singularity I would recommend pushing a bit more. Here’s a paper that the Singularity group recently published you can send to your admins: http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0177459. Here’s a list of other HPCs that are successfully running singularity: https://docs.google.com/spreadsheets/d/1Vc_1prq_1WHGf0LWtpUBY-tfKdLLM_TErjnCe1mY5m0/pub?gid=1407658660&single=true&output=pdf

I hope this helps!

Best,
Chris

Thanks Chris, I passed those details on to our HPC folks. I did try it on the HPC but immediately ran into an issue.

With regard to trying get things running on my own machine, I’m running into some very basic difficultly in that I cannot run fmriprep from the command line as you recommend. I have reinstalled both docker and fmriprep, but when I try to run e.g., fmriprep-docker --participant_label 03 --no-skull-strip-ants --no-freesurfer
~/fmrip/anderson_ADHD/BIDS/ds008 ~/fmri/anderson_ADHD/BIDS/ds008/out/out
participant
I just get “fmriprep-docker: command not found”

Is this a simple path issue? I’m sorry if so!

When I got it to work before I used the following command:
python /Users/clarekelly/Library/Python/2.7/lib/python/site-packages/fmriprep_docker.py --no-freesurfer ~/fmri/dsAndersonADHD/ ~/fmri/dsAndersonADHD/out participant -w ~/fmri/dsAndersonADHD/scratch

But now that’s not working either, I get:
Traceback (most recent call last):
File “/Users/clarekelly/Library/Python/2.7/lib/python/site-packages/fmriprep_docker.py”, line 3, in
from builtins import int, map, input, zip
ImportError: No module named builtins

Thanks for all your help,
Clare

It seems there is something incorrect with your local python install. Two options:

  1. Install anaconda and us it instead of the python shipped with OS X. After doing this and pip install fmriprep-docker things should work. Getting anaconda might be good for your other python projects.

  2. Alternatively you can forgo the python wrapper and run docker directly as described in http://fmriprep.readthedocs.io/en/stable/installation.html#docker-container

Hi Clare,

Sorry about the delay. @ChrisGorgolewski’s advice is good, but hopefully I can add to it a little. You’re describing two related issues: First, the fmriprep-docker command was not installed in your current Python environment; second, the specific error you’re getting while trying to run fmriprep_docker.py is because you don’t have future installed in that environment.

Installing fmriprep-docker will install future as a dependency, so (re-)running pip install fmriprep-docker in your current environment, as @ChrisGorgolewski suggests, should resolve both issues. If that results in a permissions issue, and you don’t want to use anaconda, you can try pip install --user fmriprep-docker.

Also, I wouldn’t suggest using fmriprep_docker.py directly; we make automatic modifications when we issue a release that ensures that it runs the correct version of fmriprep. It can still be run, using -i <docker image>, where <docker image> is something like poldracklab/fmriprep:0.4.4.

Hope this helps.

Thanks @ChrisGorgolewski and @effigies.
I installed anaconda and after some further attempts managed to get fmriprep-docker working - I had to install it using: sudo -H pip install fmriprep-docker

It’s running now, will see how far I get this time :slight_smile:

Thanks a million!
Clare

A post was split to a new topic: Running fMRIPrep Docker