Slowness running QSIRecon

Software Support
1

Summary of what happened:

Hi all,

I am running QSIRecon on outputs generated by QSIPrep. Currently, I am using the dsi_studio_autotrack pipeline. When I started running QSIRecon (in December last year), the pipeline ran in about 2 hours. Now, it takes about 9 hours to run the same pipeline on the same participant data. Does anyone know anything else I could try or if something changed in the past two weeks?

Command used (and if a helper script was used, a link to the helper script or the command generated):

Here is the docker command I am currently using:

docker run --rm --user $(id -u):$(id -g) \
-v /projects/public_warehouse/QSIPREP/dwi_runs/qsiprep_outputs:/data \
-v /projects/public_warehouse/QSIPREP/dwi_runs/qsirecon_outputs:/out \
-v /projects/public_warehouse/QSIPREP/dwi_runs/tmp:/tmp \
-v ${FREESURFER_HOME}/license.txt:/opt/freesurfer/license.txt \
pennlinc/qsirecon:latest /data /out participant \
--nthreads 6 \
--omp-nthreads 6 \
--fs-license-file /opt/freesurfer/license.txt \
--participant-label sub-NDARAA001ABC \
--recon-spec dsi_studio_autotrack \
--stop-on-first-crash
2

Hi @nathaliaesper and welcome to neurostars!

That could be due to the additional bundles that are tracked. For example, the cranial nerves take a pretty long time and are a more recent addition. You can create your own recon spec to only run bundles you want. qsirecon/qsirecon/data/pipelines/dsi_studio_autotrack.yaml at main · PennLINC/qsirecon · GitHub (change the track_id parameter. any bundle that contains any part of a listed entity will be included)

As a side note, I always recommend not using the latest tag, and instead specifying a version number, so you don’t find yourself switching versions unknowingly during a study.

Best,
Steven

3

Thank you, Steven, for your prompt reply.

So, just checking, my docker command should be like this:

docker run --rm --user $(id -u):$(id -g) \
-v /data3/cdb/nesper/Rockland_DTI/qsiprep_outputs:/data \
-v /data3/cdb/nesper/Rockland_DTI/qsirecon_outputs_adapted:/out \
-v /data3/cdb/nesper/Rockland_DTI/pipeline/dsi_studio_autotrack.yaml:/config.yaml \
-v /data3/cdb/nesper/Rockland_DTI/tmp:/tmp \
-v /data3/cdb/nesper/Rockland_DTI/work:/work \
-v ${FREESURFER_HOME}/license.txt:/opt/freesurfer/license.txt \
pennlinc/qsirecon:1.0.0 /data /out participant --work-dir /work \
--nthreads 6 \
--omp-nthreads 6 \
--fs-license-file /opt/freesurfer/license.txt \
--participant-label sub-A00081724 \
--config-file /config.yaml  \
--stop-on-first-crash
4

Hi @nathaliaesper
Not quite, the recon spec goes to the —recon-spec argument.

Did you make the necessary changes you wanted to the yaml file?

Best,
Steven

5

Thank you @Steven

And yes, I made the changes I wanted to the yaml file.
The pipeline is running.

Best,
Nathalia

1 Like
6

Hi @Steven
I ran a few tests and still have the same issue. I changed the track_id parameter so that it did not include the cranial nerves and cerebellum bundles, but even without them, the pipeline takes more than 7h to run per participant. I appreciate any other advice you may have.

This is the current Docker command I am using:

docker run --rm --user $(id -u):$(id -g) \
-v /projects/dwi_runs/qsiprep_outputs:/data \
-v /projects/dwi_runs/qsirecon_outputs:/out \
-v /projects/dwi_runs/adapted_pipeline/dsi_studio_autotrack.yaml:/config.yaml \
-v /projects/dwi_runs/tmp:/tmp \
-v /projects/dwi_runs/work:/work \
-v ${FREESURFER_HOME}/license.txt:/opt/freesurfer/license.txt \
pennlinc/qsirecon:1.0.0 /data /out participant \
--work-dir /work \
--nthreads 6 \
--omp-nthreads 6 \
--fs-license-file /opt/freesurfer/license.txt \
--participant-label ${participant_id} \
--recon-spec /config.yaml \
--stop-on-first-crash

And here is the dsi_studio_autotrack.yaml file I am using:

name: dsistudio_pipeline
nodes:
-   action: reconstruction
    input: qsirecon
    name: dsistudio_gqi
    parameters:
        method: gqi
    qsirecon_suffix: DSIStudio
    software: DSI Studio
-   action: autotrack
    input: dsistudio_gqi
    name: autotrackgqi
    parameters:
        tolerance: 22,26,30
        track_id: Association,Projection,Commissure
        track_voxel_ratio: 2.0
        yield_rate: 1.0e-06
    qsirecon_suffix: DSIStudio
    software: DSI Studio
space: T1w

QSIRecon version 1.0.0

Thank you so much for your help.

7

Hi @nathaliaesper,

Beyond that, I can only think of increasing CPU count, and setting the omp parameter to less than the total thread count (to eliminate chances that one process is using all CPUs)

Best,
Steven

8

Hi @Steven,
Is there a recommended CPU count?
Thank you for noticing the omp parameter; I changed it!

Best,
Nathalia

9

Not necessarily, but more is better (to an extent)! But powers of 2 tend to play well. Try 16?