QSIprep segmentation fault while executing node plot peaks

Software Support
1

Hi,

Summary of what happened:

QSIprep crashes during processing. I’ve tried on several subjects/datasets, all result in the same error. The same command worked just fine when using a previous version of QSIprep (0.14.3) on another server (without resource manager as is the case currently, see below)… Any help would be greatly appreciated!

Command used (and if a helper script was used, a link to the helper script or the command generated):

singularity run --cleanenv -B $BIDSDIR -B $OUTDIR -B $WORKDIR -B $FS_LICENSE -B $BIDS_CONFIG_QSI $QSI_IMG \
    $BIDSDIR $OUTDIR participant \
    --nthreads 16 \
    --skip-bids-validation \
    --hmc-model eddy \
    --hmc-transform Rigid \
    --output-space T1w \
    --output-resolution 1.2 \
    --unringing-method mrdegibbs \
    --dwi-denoise-window 5 \
    --recon-spec mrtrix_singleshell_ss3t_noACT \
    --fs-license-file $FS_LICENSE \
    --bids-filter-file $BIDS_CONFIG_QSI \
    --work-dir $WORKDIR \
    --participant-label $SUBJ_ID

Version:

0.16.1

Environment (Docker, Singularity, custom installation):

Singularity

Data formatted according to a validatable standard? Please provide the output of the validator:

Yes

Relevant log outputs (up to 20 lines):

Node: qsirecon_wf.sub-0011_mrtrix_singleshell_ss3t_noACT.sub_0011_ses_T1_acq_024_run_1_space_T1w_desc_preproc_recon_wf.ss3t_csd.plot_peaks
Working directory: /scratch/anw/lcbreedt/testLuna/noCoPipeline/qsifmriprep-workdir/qsirecon_wf/sub-0011_mrtrix_singleshell_ss3t_noACT/sub_0011_ses_T1_acq_024_run_1_space_T1w_desc_preproc_recon_wf/ss3t_csd/plot_peaks

Node inputs:

args = <undefined>
background_image = <undefined>
directions_file = <undefined>
environ = {}
fib_file = <undefined>
mask_file = <undefined>
mif_file = <undefined>
odf_file = <undefined>
odf_report = odfs_mosaic.png
odf_rois = <undefined>
peak_report = peaks_mosaic.png
peaks_only = False
subtract_iso = False

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node plot_peaks.

Cmdline:
	recon_plot --background_image /data/anw/anw-gold/MULTINET/l.breedt/testLuna/noCoPipeline/qsifmriprep-outdir/qsiprep/sub-0011/ses-T1/dwi/sub-0011_ses-T1_acq-024_run-1_space-T1w_dwiref.nii.gz --mask_file /scratch/anw/lcbreedt/testLuna/noCoPipeline/qsifmriprep-workdir/qsirecon_wf/sub-0011_mrtrix_singleshell_ss3t_noACT/sub_0011_ses_T1_acq_024_run_1_space_T1w_desc_preproc_recon_wf/qsirecon_anat_wf/resample_mask/sub-0011_desc-brain_mask_resample.nii.gz --mif /scratch/anw/lcbreedt/testLuna/noCoPipeline/qsifmriprep-workdir/qsirecon_wf/sub-0011_mrtrix_singleshell_ss3t_noACT/sub_0011_ses_T1_acq_024_run_1_space_T1w_desc_preproc_recon_wf/ss3t_csd/intensity_norm/sub-0011_ses-T1_acq-024_run-1_space-T1w_desc-preproc_dwi_wm_mtnorm.mif --odfs_image odfs_mosaic.png --odf_rois /scratch/anw/lcbreedt/testLuna/noCoPipeline/qsifmriprep-workdir/qsirecon_wf/sub-0011_mrtrix_singleshell_ss3t_noACT/sub_0011_ses_T1_acq_024_run_1_space_T1w_desc_preproc_recon_wf/qsirecon_anat_wf/odf_rois/crossing_rois_trans.nii.gz --peaks_image peaks_mosaic.png
Stdout:
	230306-17:30:17,411 nipype.workflow INFO:
		 b''
	230306-17:30:17,411 nipype.workflow INFO:
		 b''
	230306-17:30:17,413 nipype.interface INFO:
		 converting /scratch/anw/lcbreedt/testLuna/noCoPipeline/qsifmriprep-workdir/qsirecon_wf/sub-0011_mrtrix_singleshell_ss3t_noACT/sub_0011_ses_T1_acq_024_run_1_space_T1w_desc_preproc_recon_wf/ss3t_csd/intensity_norm/sub-0011_ses-T1_acq-024_run-1_space-T1w_desc-preproc_dwi_wm_mtnorm.mif to plot ODF/peaks
	230306-17:30:22,357 nipype.interface INFO:
		 saving peaks image to /scratch/anw/lcbreedt/testLuna/noCoPipeline/qsifmriprep-workdir/qsirecon_wf/sub-0011_mrtrix_singleshell_ss3t_noACT/sub_0011_ses_T1_acq_024_run_1_space_T1w_desc_preproc_recon_wf/ss3t_csd/plot_peaks/peaks_mosaic.png
	230306-17:30:22,648 nipype.interface INFO:
		 Plotting slice indices {'x': [74, 81, 87], 'y': [88, 96, 104], 'z': [75, 82, 88]}
Stderr:
	Fatal Python error: Segmentation fault

	Current thread 0x00007fbb449e82c0 (most recent call first):
	  File "/usr/local/miniconda/lib/python3.8/site-packages/fury/window.py", line 760 in record
	  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/cli/recon_plot.py", line 181 in plot_peak_slice
	  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/cli/recon_plot.py", line 201 in peak_slice_series
	  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/cli/recon_plot.py", line 115 in recon_plot
	  File "/usr/local/miniconda/bin/recon_plot", line 8 in <module>
	Segmentation fault (core dumped)
Traceback:
	Traceback (most recent call last):
	  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 454, in aggregate_outputs
	    setattr(outputs, key, val)
	  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/interfaces/base/traits_extension.py", line 330, in validate
	    value = super(File, self).validate(objekt, name, value, return_pathlike=True)
	  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/interfaces/base/traits_extension.py", line 135, in validate
	    self.error(objekt, name, str(value))
	  File "/usr/local/miniconda/lib/python3.8/site-packages/traits/base_trait_handler.py", line 74, in error
	    raise TraitError(
	traits.trait_errors.TraitError: The 'peak_report' trait of a _ReconPeaksReportOutputSpec instance must be a pathlike object or string representing an existing file, but a value of '/scratch/anw/lcbreedt/testLuna/noCoPipeline/qsifmriprep-workdir/qsirecon_wf/sub-0011_mrtrix_singleshell_ss3t_noACT/sub_0011_ses_T1_acq_024_run_1_space_T1w_desc_preproc_recon_wf/ss3t_csd/plot_peaks/peaks_mosaic.png' <class 'str'> was specified.

	During handling of the above exception, another exception occurred:

	Traceback (most recent call last):
	  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 401, in run
	    outputs = self.aggregate_outputs(runtime)
	  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 461, in aggregate_outputs
	    raise FileNotFoundError(msg)
	FileNotFoundError: No such file or directory '/scratch/anw/lcbreedt/testLuna/noCoPipeline/qsifmriprep-workdir/qsirecon_wf/sub-0011_mrtrix_singleshell_ss3t_noACT/sub_0011_ses_T1_acq_024_run_1_space_T1w_desc_preproc_recon_wf/ss3t_csd/plot_peaks/peaks_mosaic.png' for output 'peak_report' of a CLIReconPeaksReport interface

Screenshots / relevant information:

I’m running this on a cluster that utilizes Slurm for resource management. I use the batch settings detailed below, but have also tried w/ different settings to no avail.

#SBATCH --mem-per-cpu=10G             
#SBATCH --cpus-per-task=4     	     
#SBATCH --time=3-00:00:00

Best!
Lucas

2

Hi @lcbreedt and welcome to neurostars!

Try adding the --skip-odf-reports option.

Unrelated, but the HSVS tractography recon specs substantially improve tractography compared to *_noACT, so I’d recommend using them. This would mean running freesurfer on your T1 data and you’d need to have some kind of SDC for your DWI (you can use the fieldmapless option if you do not have a fieldmap), but I think it is worth it.

Best,
Steven

3

Hi @Steven,

Thanks-- and thank you for the very quick response! I’ll try it and report back.

Do you happen to also know what the cause of this issue might be though? And what (if any) consequences are there of adding this flag?

Also, I appreciate the advice; however, I’m working with glioma patient data on which freesurfer doesn’t work very well, limiting my options.

Best!
Lucas

4

Hi Lucas,

Sometimes some systems have trouble rendering the peaks plot which is used in the ODF report part of the QSIRecon html. Skipping this does not have any impact on the final image/tractography, it just means part of the summary html will be missing. You can still load your WM FOD image into something like MRView to do your own quality assurance.

Got it.

Best,
Steven

5

Just wanted to report back and say this worked-- thanks again!

Best,
Lucas

1 Like