Issues with QSIPrep output and HTML report for DWI data from Philips 3T scanner

Software Support
1

Summary of What Happened

Hello everyone,

I have been facing issues with running QSIPrep on DWI data collected from a Philips 3T scanner, and I am hoping for some guidance to resolve it. I have tried processing both concatenated and non-concatenated DWI files, but I encounter issues with the output structure, including missing HTML reports and what seems like an infinite loop during processing. Additionally, the BIDS validator outputs some warnings that might be relevant.

Command

1. Non-concatenated DWI files (Without Susceptibility Distortion Correction):

For the first attempt, I ran QSIPrep with two separate DWI files and disabled susceptibility distortion correction using the --ignore-topup flag. Here’s the command I used:

singularity run --bind /path/to/license.txt:/opt/freesurfer/license.txt \
--cleanenv /path/to/qsiprep.sif \
/path/to/data /path/to/output \
participant \
--skip_bids_validation \
--separate-all-dwis \
--ignore-topup \
--output-space T1w \
--n_cpus 2 \
--omp-nthreads 2 \
--mem_mb 16000 \
-w /path/to/work \
--verbose \
--output-resolution 1.5

2. Concatenated DWI file (Also Without Susceptibility Distortion Correction):

I also tried concatenating the two DWI scans into a single file and re-ran QSIPrep without susceptibility distortion correction (--ignore-topup), this time using Apptainer:

apptainer run --cleanenv \
-B /path/to/data:/data:ro \
-B /path/to/output:/out \
-B /path/to/license.txt:/opt/freesurfer/license.txt \
/path/to/qsiprep.sif \
/data /out participant \
--fs-license-file /opt/freesurfer/license.txt \
--output-resolution 1.5 \
--n_cpus 4 \
--omp-nthreads 4 \
--mem_mb 8000 \
--ignore fieldmaps

Version

  • QSIPrep version: v0.23.1.dev0+g634483f.d20240830
  • Apptainer version: 1.3.0

Environment

I am running QSIPrep using Singularity/Apptainer containers.

Data Format and BIDS Validation Output

The data structure includes:

  • /data/sub-XX/ses-01/anat – T1-weighted anatomical images.
  • /data/sub-XX/ses-01/dwi – Contains the two separate DWI runs (for the non-concatenated attempt) or the concatenated DWI file.
  • /data/sub-XX/ses-01/fmap – Includes AP and PA fieldmaps with IntendedFor fields pointing to the DWI file.
  • /data/sub-XX/ses-01/func – Functional MRI files.

In the fieldmap JSON files, I have specified the IntendedFor field as follows:

  • AP fieldmap JSON: "IntendedFor": "ses-01/dwi/sub-22_ses-01_dir-AP_dwi.nii.gz"
  • PA fieldmap JSON: "IntendedFor": "ses-01/dwi/sub-22_ses-01_dir-AP_dwi.nii.gz"

I ran the BIDS validator and received the following warnings:

1: [WARN] You should define 'SliceTiming' for this file. If you don't provide this information slice time correction will not be possible.
   ./sub-22/ses-01/func/sub-22_ses-01_task-rest_run-01_bold.nii.gz
   ./sub-22/ses-01/func/sub-22_ses-01_task-rest_run-02_bold.nii.gz

2: [WARN] Not all subjects/sessions/runs have the same scanning parameters.
   ./sub-22/ses-01/func/sub-22_ses-01_task-rest_run-02_bold.nii.gz

Problem Description and Observations

1. Loop or No Output when Running Without Susceptibility Distortion Correction:

In the run with non-concatenated DWI files, with susceptibility distortion correction disabled (--ignore-topup), the QSIPrep process seems to enter an infinite loop and does not produce a final HTML report. The only HTML outputs generated are in the figures folder, which lack comprehensive information on processing errors or completed steps.

2. Error with Susceptibility Distortion Correction Enabled (topup):

When I enabled topup for susceptibility distortion correction and used the concatenated DWI file, the pipeline produced an HTML report but returned an error in the node gather_inputs. Here’s the error message (screenshot attached):

  • Node Name: qsiprep_0_23_wf.sub_22_wf.dwi_preproc_ses_01_dir_AP_wf.hmc_sdc_wf.gather_inputs
  • Error:

image
image1142×673 174 KB

  • The inputs listed include paths to eddy_params.json and epi_fmaps, which appear to be correctly specified. However, the process fails, potentially due to a configuration or compatibility issue.

3. BIDS Validation Warnings:

The BIDS validator outputs warnings about missing SliceTiming information in the functional files, which could affect slice timing correction, and about inconsistent parameters across runs. I’m uncertain if these issues are contributing to the QSIPrep processing problems.

Relevant Log Outputs

Here is a portion of the error traceback when running with the concatenated DWI file and topup (susceptibility distortion correction) enabled:

Traceback (most recent call last):
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node gather_inputs.

Screenshot / Additional Information

Please see the attached screenshot for more details on the error message.

Questions:

  1. Has anyone experienced similar issues with QSIPrep on Philips 3T data, particularly with susceptibility distortion correction?
  2. Could the BIDS validation warnings (e.g., missing SliceTiming information and inconsistent parameters) be contributing to the QSIPrep issues?
  3. Are there specific QSIPrep parameters or setup adjustments that might help resolve the looping issue or help generate the missing HTML report?

Any insights or suggestions to troubleshoot this would be greatly appreciated. Please let me know if you have any additional questions. Thank you so much!

2

Hi @aritban,

You shouldn’t concatenate the images, you should let QSIPrep take care of that internally. Are your two images the same phase encoding direction?

No, the slice timing is just relevant for BOLD, not DWI.

This isn’t a flag, according to the documentation (Usage — qsiprep 0.24.1.dev1+g74f7058 documentation). --ignore fieldmaps is an argument though.

There should be more in the traceback that describes the precise error. Do you have anything else you can provide?

You can also try the recently released 0.24.0 (just keep in mind starting in this version, you need to use argument names with dashes, not underscores (like mem-mb instead of mem_mb).

Best,
Steven

3

Hi Steven,

To answer your earlier question (> There should be more in the traceback that describes the precise error. Do you have anything else you can provide?), I have this:

image
image1401×932 68.4 KB
.

Thank you for your feedback and suggestions. I attempted several runs of QSIPrep with different configurations and steps, but the results have been incomplete. Let me walk you through the details and observations:

Steps I Tried:

  1. Anatomical Data Only
    Used the --anat-only flag to process the anatomical data (T1-weighted).
    2.Outcome: Ran without showing any errors but the expected outputs were not generated.

DWI Without Susceptibility Distortion Correction
Used the --ignore-fieldmaps flag for DWI processing, skipping susceptibility correction.
Command Used:

nohup singularity run --bind /path/to/license.txt:/opt/freesurfer/license.txt \
--cleanenv /path/to/qsiprep_0.24.0.sif \
/path/to/data \
/path/to/output \
participant \
--skip-bids-validation \
--separate-all-dwis \
--ignore-fieldmaps \
--output-space T1w \
--n-cpus 2 \
--omp-nthreads 2 \
--mem-mb 16000 \
-w /path/to/work_dir \
--output-resolution 1.5 \
--verbose > qsiprep_nosuscorr.log 2>&1 &

Outcome:
The pipeline completed but did not generate the expected outputs. The directory structure included:

/logs/
/sub-22/
    /figures/
    /log/
    /ses-01/dwi/

Inside the ses-01/dwi/ folder, I only found:

  • sub-22_ses-01_dir-AP_confounds.tsv
  • sub-22_ses-01_dir-AP_desc-SliceQC_dwi.json

There were no preprocessed DWI images, bvec, bval files, or a comprehensive HTML report summarizing the processing.

  1. DWI Without Eddy Correction
    Used the --dwi-no-eddy flag.
    2.Command Used:*
nohup singularity run --bind /path/to/license.txt:/opt/freesurfer/license.txt \
--cleanenv /path/to/qsiprep_0.24.0.sif \
/path/to/data \
/path/to/output_noeddy \
participant \
--skip-bids-validation \
--separate-all-dwis \
--ignore-fieldmaps \
--output-space T1w \
--n-cpus 2 \
--omp-nthreads 2 \
--mem-mb 16000 \
-w /path/to/work_dir_noeddy \
--output-resolution 1.5 \
--dwi-no-eddy \
--verbose > qsiprep_noeddy.log 2>&1 &

3.Outcome:
Similar to the above. The results were limited to QC and confounds files but lacked preprocessed DWI images and summary reports.

Observations:

  1. The processing pipeline does not seem to generate errors but also does not produce the expected outputs for DWI processing.
  2. The figures/ folder contains diagnostic plots such as carpet plots, denoising diagnostics, and partial summary HTML files (e.g., sub-22_summary.html), but these are not comprehensive and do not indicate a complete pipeline execution.
  3. The log/ folder includes log files, but there are no detailed error messages or traces indicating why the processing is incomplete.

More observations:

  1. Concatenation of DWI Files
    Based on your earlier suggestion, I avoided concatenating the DWI files in subsequent runs and allowed QSIPrep to handle them internally. However, the issue persisted.
  2. Incomplete or Missing Outputs
    The missing outputs could indicate:
  • An issue in the intermediate pipeline stages (e.g., preprocessing nodes not running fully).
  • A configuration mismatch in the QSIPrep arguments.

Thank you for your guidance. If needed, I can provide the exact log contents or traceback details for further analysis. Looking forward to your advice on how to proceed so as to get valid qsiprep generated outputs.

Best,
Arit

4

Hi @aritban,

It looks like there is no TotalReadoutTime in your dwi JSONs. Can you confirm that, and if so, get the value from your scanner protocol?

The proper syntax is --ignore fieldmaps. Please refer to the documentation: Usage — qsiprep 1.0.0rc2.dev7+g8bb6c0d documentation. I also don’t recommend ignoring them if you have them.

This is no longer an argument.

Is there a reason you are not having QSIPrep concatenate the data? The only use case I can think of is if you are testing run1 vs run2 within subject. Otherwise, having a single more dense DWI image should lead to better postprocessed derivatives.

I don’t think this is a flag either. Also, it is not recommended to skip eddy.

If you have a log from your job scheduler that would help. But providing the TotalReadoutTime in your dwi json should get past the pictured error.

Best,
Steven

5

Hi @Steven ,

It looks like there is no TotalReadoutTime in your dwi JSONs. Can you confirm that, and if so, get the value from your scanner protocol?

I do have TotalReadoutTime in my DWI JSONs. Do I have to put the value (“TotalReadoutTime”: 0.142664) in the code itself?

–ignore-fieldmaps: that was a typing mistake on my part, I do have --ignore fieldmaps

Is there a reason you are not having QSIPrep concatenate the data? The only use case I can think of is if you are testing run1 vs run2 within subject. Otherwise, having a single more dense DWI image should lead to better postprocessed derivatives.

I thought the documentation says,
“–separate-all-dwis:
don’t attempt to combine dwis from multiple runs. Each will be processed separately”

But I hear what you say, so I will delete than flag.

If you have a log from your job scheduler that would help:

I only have these files from the figures folder that was generated which I received as outputs:

image
image1811×94 15.7 KB

image

image
image681×511 40.1 KB

image
image818×440 36.6 KB

image
image937×234 25 KB

After making the modifications I have now,

nohup singularity run --bind pathto/license.txt:/opt/freesurfer/license.txt
–cleanenv ~/bin/qsiprep.sif
path to input data/data
path to output data/qsiprep_output_nosuscorr
participant
–skip-bids-validation
–ignore fieldmaps
–output-resolution 1.5
–nprocs 4
–omp-nthreads 2
–mem 16000
-w /path to output/qsiprep_work_nosuscorr
–verbose > qsiprep_nosuscorr.log 2>&1 &

Please let me know if you have any questions.

Best,
Arit

6

Hi @aritban,

I would suggest ultimately trying to get the fieldmaps to work so you can correct for susceptability distortions.

Can you provide the contents a json? Do you have the readout time in the fieldmaps?

If you are running on a cluster job scheduler, do you not also have the resulting scheduler log files? E.g., a SLURM .out / .err file?

Best,
Steven