Canonical way to use fMRIprep's `--level` flag

Software Support
1

Summary of what happened:

Hi, I have recently learned about fMRIprep’s new --level flag and I am currently testing it on a toy dataset. What I would like to do is the following:

  1. Run fMRIprep in the minimal mode.
  2. Run fMRIprep again in the full mode, so that it reuses the derivatives precomputed in the minimal step and generates all target outputs.

I was able to successfully run the first step, but I’m struggling to find the right command call for the second step.

For the second step, I was also trying to use the --derivatives flag, but so far without success.

What command should I use for the second step, provided that I would like all the target outputs to be saved as if fMRIprep was run normally (without specifying --level flag).

I’m pinging @effigies, in the hope that he’ll be able to help. :slightly_smiling_face:

Command used (and if a helper script was used, a link to the helper script or the command generated):

Here’s my fMRIprep call for the first step:

fmriprep-docker \
./ds000102 \
./ds000102/derivatives \
participant \
    --participant-label 01 \
    --fs-license-file ./license.txt \
    --fs-no-reconall \
    --level minimal \
    --skip-bids-validation \
    --ignore slicetiming

For the second step I have tried many different variants, but I’m not sure what’s the right command call.

Version:

v23.2.0

Environment (Docker, Singularity / Apptainer, custom installation):

I’m using a fmriprep-docker wrapper, installed as recommended in the documentation.

I’m using a MacOS machine.

Data formatted according to a validatable standard? Please provide the output of the validator:

The data has been obtained from OpenNeuro:

cd ~/playground
git clone https://github.com/OpenNeuroDatasets/ds000102.git
cd ds000102
datalad get sub-01

Relevant log outputs (up to 20 lines):

PASTE LOG OUTPUT HERE

Screenshots / relevant information:

2

I would do it this way:

Stage 1:

fmriprep-docker \
./ds000102 \
./ds000102/derivatives/fmriprep-minimal \
participant \
    --participant-label 01 \
    --fs-license-file ./license.txt \
    --fs-no-reconall \
    --level minimal \
    --skip-bids-validation \
    --ignore slicetiming

Stage 2:

fmriprep-docker \
./ds000102 \
./ds000102/derivatives/fmriprep-full \
participant \
    -d ./ds000102/derivatives/fmriprep-minimal \
    --participant-label 01 \
    --fs-license-file ./license.txt \
    --fs-no-reconall \
    --skip-bids-validation \
    --ignore slicetiming
3

Thank you for getting back to me so quickly, @effigies!

I have created a fresh example dataset, to make sure that I use the same directory structure as recommended above.

I was able to run Stage 1 without any problems, but when trying to run Stage 2, I get the following error:

% fmriprep-docker \
> ./ds000102 \
> ./ds000102/derivatives/fmriprep-full \
> participant \
>     -d ./ds000102/derivatives/fmriprep-minimal \
>     --participant-label 01 \
>     --fs-license-file ./license.txt \
>     --fs-no-reconall \
>     --skip-bids-validation \
>     --ignore slicetiming
Traceback (most recent call last):
  File "/Users/cynamon/.pyenv/versions/fair/bin/fmriprep-docker", line 8, in <module>
    sys.exit(main())
  File "/Users/cynamon/.pyenv/versions/3.10.13/envs/fair/lib/python3.10/site-packages/fmriprep_docker/__main__.py", line 420, in main
    opts, unknown_args = parser.parse_known_args()
  File "/Users/cynamon/.pyenv/versions/3.10.13/lib/python3.10/argparse.py", line 1866, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/Users/cynamon/.pyenv/versions/3.10.13/lib/python3.10/argparse.py", line 2079, in _parse_known_args
    start_index = consume_optional(start_index)
  File "/Users/cynamon/.pyenv/versions/3.10.13/lib/python3.10/argparse.py", line 2019, in consume_optional
    take_action(action, args, option_string)
  File "/Users/cynamon/.pyenv/versions/3.10.13/lib/python3.10/argparse.py", line 1943, in take_action
    action(self, namespace, argument_values, option_string)
  File "/Users/cynamon/.pyenv/versions/3.10.13/envs/fair/lib/python3.10/site-packages/fmriprep_docker/__main__.py", line 266, in __call__
    k, v = kv.split("=")
ValueError: not enough values to unpack (expected 2, got 1)
4

Okay, this looks like an inconsistency between fMRIPrep and the fmriprep-docker wrapper. In the wrapper, I implemented -d derivative-name=derivative-location, while in the main app I only implemented -d derivative-location.

You should be able to use -d minimal=./ds000102/derivatives/fmriprep-minimal.

5

Thank you, @effigies, it worked as expected!

I have one follow-up question though. From the existing documentation I have learned that the idea behind introducing the --level and --derivatives flags was to separate the typical fMRIprep pipeline into two stages:

  • the “fit stage”, in which only the most essential derivatives are generated
  • the “transform stage”, in which the remanining derivatives can be generated determininstically at a minimal computational cost.

I have run fMRIprep twice with -d flag to reuse the same (precomputed) derivatives. However, it seems that some of the derivatives generated in those two separate runs are different (to be precise, they are not bit identical). Is this something to be expected?

6

The goal is for there not to be differences, as the minimal derivatives should not require any new calculations, just deterministic application. We may not have achieved that yet. Please feel free to post issues with which derivatives are differing.

1 Like
7

Will do, thank you for your time!