QSIPrep issue post "extract_b0_series"/"split_eddy_lps"

pierre-nedelec · February 20, 2025, 9:12pm

Summary of what happened:

QSIPrep fails after the node “extract_b0_series” or “split_eddy_lps”. I’m running it with HARDI images acquired in AP/PA:

├── anat
│   ├── sub-testDTI_ses-multiHARDI_T1w.json
│   └── sub-testDTI_ses-multiHARDI_T1w.nii
└── dwi
    ├── sub-testDTI_ses-multiHARDI_dir-AP_dwi.bval
    ├── sub-testDTI_ses-multiHARDI_dir-AP_dwi.bvec
    ├── sub-testDTI_ses-multiHARDI_dir-AP_dwi.json
    ├── sub-testDTI_ses-multiHARDI_dir-AP_dwi.nii.gz
    ├── sub-testDTI_ses-multiHARDI_dir-PA_dwi.bval
    ├── sub-testDTI_ses-multiHARDI_dir-PA_dwi.bvec
    ├── sub-testDTI_ses-multiHARDI_dir-PA_dwi.json
    └── sub-testDTI_ses-multiHARDI_dir-PA_dwi.nii.gz

With eddy_params.json:

{
    "flm": "quadratic",
    "slm": "none",
    "fep": false,
    "interp": "spline",
    "nvoxhp": 1000,
    "fudge_factor": 10,
    "dont_sep_offs_move": false,
    "dont_peas": false,
    "niter": 5,
    "method": "lsr",
    "repol": true,
    "num_threads": 1,
    "is_shelled": true,
    "use_cuda": true,
    "cnr_maps": true,
    "residuals": false,
    "output_type": "NIFTI_GZ",
    "args": "--fwhm=10,0,0,0,0 "
}

Command used (and if a helper script was used, a link to the helper script or the command generated):

apptainer run --cleanenv --containall --writable-tmpfs --nv \
    --env CUDA_VISIBLE_DEVICES=$CUDA_VISIBLE_DEVICES \
    -B ${DERIVATIVES_DIR_TMP}:/derivatives:ro,${DERIVATIVES_DIR_TMP}:/out,${WORK_DIR}:${WORK_DIR},${FS_LICENSE}:/opt/freesurfer/license.txt \
    -B $EDDY_CONFIG:$EDDY_CONFIG \
   /software/qsiprep_${VERSION}.sif \
    /derivatives/nii /out/qsiprep participant \
    --fs-license-file /opt/freesurfer/license.txt \
    --output-resolution $OUTPUT_RES \
    --work-dir ${WORK_DIR} \
    --participant-label ${subjectID} \
    --session-id ${sessionID} \
    --subject-anatomical-reference sessionwise \
    --skip-anat-based-spatial-normalization \
    --denoise-method none \
    --eddy-config $EDDY_CONFIG \
    --nthreads ${ncpus} \
    --omp-nthreads $((ncpus - 2)) \
    --stop-on-first-crash \
    -v -v

Version:

1.0.0rc2.dev0+g789be41.d20241119

Environment (Docker, Singularity / Apptainer, custom installation):

Singularity

Relevant log outputs (up to 20 lines):

250220-11:50:35,425 nipype.workflow INFO:
		[Node] Finished "extract_b0_series", elapsed time 2.386315s.
250220-11:50:35,425 nipype.workflow WARNING:
		Storing result file without outputs
250220-11:50:35,426 nipype.workflow WARNING:
		[Node] Error on "qsiprep_1_0_wf.sub_testDTI_ses_multiHARDI_wf.dwi_preproc_ses_multiHARDI_wf.hmc_sdc_wf.extract_b0_series" (/scratch/pnedelec/qsiprep_sub-testDTI_ses-multiHARDI/qsiprep_1_0_wf/sub_testDTI_ses_multiHARDI_wf/dwi_preproc_ses_multiHARDI_wf/hmc_sdc_wf/extract_b0_series)
250220-11:50:36,832 nipype.workflow ERROR:
		Node extract_b0_series failed to run on host pgpudev1.wynton.ucsf.edu.
250220-11:50:36,841 nipype.workflow ERROR:
		Saving crash info to /out/qsiprep/sub-testDTI/log/20250220-111113_f562afb5-b446-483d-9d97-d47f9c4dd194/crash-20250220-115036-pnedelec-extract_b0_series-5a9b73e5-08e7-4c30-9b13-d9f119cd2e8f.txt
Traceback (most recent call last):
	File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py", line 66, in run_node
	result["result"] = node.run(updatehash=updatehash)
	File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 525, in run
	result = self._run_interface(execute=True)
	File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 643, in _run_interface
	return self._run_command(execute)
	File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 769, in _run_command
	raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node extract_b0_series.

Traceback:
	Traceback (most recent call last):
		File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 401, in run
		runtime = self._run_interface(runtime)
		File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsiprep/interfaces/gradients.py", line 327, in _run_interface
		new_data = nim.index_img(self.inputs.dwi_series, indices)
		File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nilearn/image/image.py", line 669, in index_img
		return _index_img(imgs, index)
		File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nilearn/_utils/niimg_conversions.py", line 77, in _index_img
		img, _get_data(img)[:, :, :, index], img.affine, copy_header=True
	IndexError: index 65 is out of bounds for axis 3 with size 65


250220-11:50:36,869 nipype.workflow CRITICAL:
		QSIPrep failed: Traceback (most recent call last):
	File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py", line 66, in run_node
	result["result"] = node.run(updatehash=updatehash)
	File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 525, in run
	result = self._run_interface(execute=True)
	File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 643, in _run_interface
	return self._run_command(execute)
	File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 769, in _run_command
	raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node extract_b0_series.

Traceback:
	Traceback (most recent call last):
		File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 401, in run
		runtime = self._run_interface(runtime)
		File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsiprep/interfaces/gradients.py", line 327, in _run_interface
		new_data = nim.index_img(self.inputs.dwi_series, indices)
		File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nilearn/image/image.py", line 669, in index_img
		return _index_img(imgs, index)
		File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nilearn/_utils/niimg_conversions.py", line 77, in _index_img
		img, _get_data(img)[:, :, :, index], img.affine, copy_header=True
	IndexError: index 65 is out of bounds for axis 3 with size 65


250220-11:50:58,14 nipype.interface INFO:
	 Converting oblique-image bvecs to world coordinate reference frame
250220-11:50:58,35 nipype.interface INFO:
	 mrinfo -dwgrad -fslgrad /scratch/pnedelec/qsiprep_sub-testDTI_ses-multiHARDI/qsiprep_1_0_wf/sub_testDTI_ses_multiHARDI_wf/dwi_preproc_ses_multiHARDI_wf/hmc_sdc_wf/split_eddy_lps/_0000eddy_corrected_LPS_fix.eddy_rotated_bvecs /scratch/pnedelec/qsiprep_sub-testDTI_ses-multiHARDI/qsiprep_1_0_wf/sub_testDTI_ses_multiHARDI_wf/dwi_preproc_ses_multiHARDI_wf/hmc_sdc_wf/split_eddy_lps/sub-testDTI_ses-multiHARDI_merged_0000.bval /scratch/pnedelec/qsiprep_sub-testDTI_ses-multiHARDI/qsiprep_1_0_wf/sub_testDTI_ses_multiHARDI_wf/dwi_preproc_ses_multiHARDI_wf/hmc_sdc_wf/split_eddy_lps/vol0000.nii.gz
250220-11:50:58,102 nipype.interface INFO:
.... more mrinfo log ....
250220-11:51:02,671 nipype.interface INFO:
		mrinfo -dwgrad -fslgrad /scratch/pnedelec/qsiprep_sub-testDTI_ses-multiHARDI/qsiprep_1_0_wf/sub_testDTI_ses_multiHARDI_wf/dwi_preproc_ses_multiHARDI_wf/hmc_sdc_wf/split_eddy_lps/_0064eddy_corrected_LPS_fix.eddy_rotated_bvecs /scratch/pnedelec/qsiprep_sub-testDTI_ses-multiHARDI/qsiprep_1_0_wf/sub_testDTI_ses_multiHARDI_wf/dwi_preproc_ses_multiHARDI_wf/hmc_sdc_wf/split_eddy_lps/sub-testDTI_ses-multiHARDI_merged_0064.bval /scratch/pnedelec/qsiprep_sub-testDTI_ses-multiHARDI/qsiprep_1_0_wf/sub_testDTI_ses_multiHARDI_wf/dwi_preproc_ses_multiHARDI_wf/hmc_sdc_wf/split_eddy_lps/vol0064.nii.gz
250220-11:51:02,725 nipype.workflow INFO:
		[Node] Finished "split_eddy_lps", elapsed time 29.704356s.
250220-11:51:02,725 nipype.workflow WARNING:
		Storing result file without outputs
250220-11:51:02,725 nipype.workflow WARNING:
		[Node] Error on "qsiprep_1_0_wf.sub_testDTI_ses_multiHARDI_wf.dwi_preproc_ses_multiHARDI_wf.hmc_sdc_wf.split_eddy_lps" (/scratch/pnedelec/qsiprep_sub-testDTI_ses-multiHARDI/qsiprep_1_0_wf/sub_testDTI_ses_multiHARDI_wf/dwi_preproc_ses_multiHARDI_wf/hmc_sdc_wf/split_eddy_lps)

Seems like extract_b0_series has an index error…
Both dwis have 1 b0 and 64 b2000. AP has json: "PhaseEncodingDirection": "j-", PA “j”.

Not sure how to go about solving this… Thanks!

Steven · February 20, 2025, 9:36pm

Hi @pierre-nedelec,

Can you confirm this behavior continues on 1.0.0 in a fresh working dir? Also is this subject specific or for everyone?

Best,
Steven

pierre-nedelec · February 20, 2025, 11:09pm

Thanks for your reply Steven!

I just reran after deleting the previous working directory and downloading the latest version (QSIPrep version 1.0.1.dev0+gee9aa2e.d20250115). Same exact errors.

I only have one scan as we’re testing our new scanners/protocols. I’m trying to compare this with a single phase acquisition but with epi fieldmaps, for which I ran a slightly different eddy and which worked fine.

mattcieslak · February 21, 2025, 1:14pm

Does the data pass BIDS validation? That part of the code is extracting the b=0s from the image - maybe the bvals don’t match?

pierre-nedelec · February 21, 2025, 6:35pm

Hi Matt! thanks for checking in! I believe so:

bids-validator@1.8.4

	e[33m1: [WARN] The recommended file /README is missing. See Section 03 (Modality agnostic files) of the BIDS specification. (code: 101 - README_FILE_MISSING)e[39m

e[36m	Please visit https://neurostars.org/search?q=README_FILE_MISSING for existing conversations about this issue.e[39m

	e[33m2: [WARN] The Authors field of dataset_description.json should contain an array of fields - with one author per field. This was triggered based on the presence of only one author field. Please ignore if all contributors are already properly listed. (code: 102 - TOO_FEW_AUTHORS)e[39m

e[36m	Please visit https://neurostars.org/search?q=TOO_FEW_AUTHORS for existing conversations about this issue.e[39m

        e[34me[4mSummary:e[24me[39m                  e[34me[4mAvailable Tasks:e[24me[39m        e[34me[4mAvailable Modalities:e[24me[39m 
        11 Files, 291.76MB                                MRI                   
        1 - Subject                                                             
        1 - Session                                                             


e[36m	If you have any questions, please post on https://neurostars.org/tags/bids.e[39m
250220-13:57:58,446 nipype.workflow INFO:
	 
Group	Subject	Sessions
0	testDTI	multiHARDI

pierre-nedelec · February 24, 2025, 8:50pm

Any thought @mattcieslak or @Steven ? Or other things I could do to understand what’s going on? It could be linked to the specific Eddy configuration for that kind of images maybe…
Thank you!

pierre-nedelec · March 3, 2025, 7:54pm

Hi @Steven and @mattcieslak ,

TL;DR

Can QSIPrep handle the Eddy LSR method, where several dwi series are merged and not concatenated?

Details

I did some more digging, and here’s where I get.
QSIPrep log says (see above for context):

[Node] Error on "qsiprep_1_0_wf.sub_testDTI_ses_multiHARDI_wf.dwi_preproc_ses_multiHARDI_wf.hmc_sdc_wf.extract_b0_series" (/scratch/pnedelec/qsiprep/qsiprep_sub-testDTI_ses-multiHARDI/qsiprep_1_0_wf/sub_testDTI_ses_multiHARDI_wf/dwi_preproc_ses_multiHARDI_wf/hmc_sdc_wf/extract_b0_series)

So I went in the report in that working (/scratch) folder, in _report/report.rst.

report.rst

Node: sub_testDTI_ses_multiHARDI_wf (dwi_preproc_ses_multiHARDI_wf (hmc_sdc_wf (extract_b0_series (gradients)
=============================================================================================================


 Hierarchy : qsiprep_1_0_wf.sub_testDTI_ses_multiHARDI_wf.dwi_preproc_ses_multiHARDI_wf.hmc_sdc_wf.extract_b0_series
 Exec ID : extract_b0_series


Original Inputs
---------------


* b0_indices : <undefined>
* b0_threshold : 100
* bval_file : /scratch/pnedelec/qsiprep/qsiprep_sub-testDTI_ses-multiHARDI/qsiprep_1_0_wf/sub_testDTI_ses_multiHARDI_wf/dwi_preproc_ses_multiHARDI_wf/pre_hmc_wf/rpe_concat/sub-testDTI_ses-multiHARDI_merged.bval
* dwi_series : /scratch/pnedelec/qsiprep/qsiprep_sub-testDTI_ses-multiHARDI/qsiprep_1_0_wf/sub_testDTI_ses_multiHARDI_wf/dwi_preproc_ses_multiHARDI_wf/hmc_sdc_wf/back_to_lps/eddy_corrected_LPS.nii.gz

The bval file is 130 long, 1 b0, 64 b2000, 1b0, 64 b2000. So, it’s the concatenated version of the two original bval files.
The nifti however is the output of Eddy LSR method, which merges the two dwi series together, so it’s only has 65 total directions (1 b0 and 64 b2000).

So when the next qsiprep step tries to open the image at position 66 (which it assumes is the 2nd b0 in a world where the images have been concatenated), it fails:

IndexError: index 65 is out of bounds for axis 3 with size 65

(65 in python indices)

Thank you for your help!

mattcieslak · March 4, 2025, 1:13am

AHA! great catch. As far as I know you are the first person to try this in qsiprep. I’m not sure how complicated the fix will be, but we should definitely support it.

pierre-nedelec · March 4, 2025, 6:42pm

Ok! so what is the next step? should I open an issue on github?

also – curious how other people have been processing this type of data with qsiprep if at all.