Very long run time on Mac (fmriprep-docker)

Software Support
1

Summary of what happened:

Running fmriprep-docker on a Mac Studio (M1 Max: 10 Core CPU and 32GB RAM; macOS Ventura 13.1) for a single subject takes a very long time.

E.g.:
Subject with 1 T1 weighted structural image, 1 EPI (>820 volumes), 1 phasedifference fieldmap (acquired with a Siemens scanner) takes ~ 16h without freesurfer surface reconstruction and ~ 3-4 days with freesurfer reconstruction.
Subject with 1 T1 weighted strucutral image, 8 EPIs (>190 volumes each), 1 phasedifference fieldmap (acquired with a Siemens scanner) takes ~ 26h without freesurfer surface reconstruction.

This is nowhere close to the expected run time mentioned in the docs (FAQ, Tips, and Tricks — fmriprep version documentation).
We have > 150 Subjects to be preprocessed and at this rate it would take several months to complete (currently we do not have access to a HPC). Therefore, I am wondering whether there are any options that I am missing that might help accelerate fmriprep?

Thank you very much for any help in advance!

Best,
Tibor

Command used (and if a helper script was used, a link to the helper script or the command generated):

    participant_label=participant01
    max_mem_mb=24000
    max_cores_cpu=8
    freesurfer_file=~/freesurfer_license.txt

    sourcepath=/path_to/sourcedata
    preprocpath=/path_to/derivatives/preproc
    workdirpath=/path_to/derivatives/workdir

    fmriprep-docker ${sourcepath} ${preprocpath} \
        participant \
        --participant-label ${participant_label} \
        --fs-license-file ${freesurfer_file} \
        --nthreads ${max_cores_cpu} \
        --stop-on-first-crash \
        --mem-mb ${max_mem_mb} \
        -v \
        -w ${workdirpath} \
        --dummy-scans 3 \
        --fs-no-reconall

Version:

Tried on both v20.2.7 and v23.1.4

Environment (Docker, Singularity, custom installation):

Docker v4.21.1 with 8 CPU cores and 27 GB RAM assigned.

Data formatted according to a validatable standard? Please provide the output of the validator:

Data is Bids valid.

2

Hi @tiborst,

You can try Brainlife.io to parallelize processing over the cloud.

Best,
Steven

3

This just isn’t going to work without someone contributing a Docker image and build process for the M1 architecture. You may have some success installing fMRIPrep in a conda environment instead of Docker:

$ git clone https://github.com/nipreps/fmriprep.git
$ cd fmriprep
$ git checkout $VERSION
$ conda env create -f env.yml
$ conda activate fmriprep
$ pip install -r requirements.txt .

If you inspect the Dockerfile, you should be able to see all of the other tools that need installing, such as AFNI, FreeSurfer, Connectome Workbench, etc. Note that any of these tools might also not be compiled for the M1/M2 architecture, so I can’t guarantee any performance.

We also can only give limited support for “bare-metal” installations, so taking this route is accepting some responsibility for figuring things out. I definitely cannot help with reproducing bugs on a Mac.

4

Dear @tiborst,

I am currently working on brining our www.Neurodesk.org project to Apple Silicon and one thing I found with respect to performance is that activating Rosetta for x86 images brings a 5 fold speedup in my tests and I haven’t seen a downside yet:

You find this under “Features in development”:

image
image2722Ă—1102 327 KB

I hope that helps
Kind regards
Steffen

1 Like