QSIPrep pipeline error

Hi everyone,

QSIPrep gives this error to me, I wonder about that but I think that refers to input data and JSON file information. I would appreciate it if you could help me. I also attached JSON file.
sub-2001_ses-baseline_run-1_dwi.txt (2.0 KB)

220928-08:59:13,497 nipype.workflow INFO:
	 [Node] Setting-up "qsiprep_wf.single_subject_2001_wf.dwi_finalize_ses_baseline_wf.transform_dwis_t1.cnr_image_type" in "/flywheel/v0/work/qsiprep_wf/single_subject_2001_wf/dwi_finalize_ses_baseline_wf/transform_dwis_t1/cnr_image_type".
220928-08:59:13,498 nipype.workflow INFO:
	 [Node] Finished "fdisp", elapsed time 0.003251s.
220928-08:59:13,499 nipype.workflow INFO:
	 [Node] Finished "add_motion_headers", elapsed time 0.003998s.
220928-08:59:13,501 nipype.workflow INFO:
	 [Job 98] Completed (qsiprep_wf.single_subject_2001_wf.dwi_preproc_ses_baseline_wf.confounds_wf.add_motion_headers).
220928-08:59:13,502 nipype.workflow INFO:
	 [Node] Executing "cnr_image_type" <qsiprep.interfaces.ants.GetImageType>
220928-08:59:13,580 nipype.workflow INFO:
	 [Node] Finished "cnr_image_type", elapsed time 0.076561s.
220928-08:59:15,357 nipype.workflow INFO:
	 [Job 96] Completed (qsiprep_wf.single_subject_2001_wf.dwi_finalize_ses_baseline_wf.transform_dwis_t1.cnr_image_type).
220928-08:59:15,358 nipype.workflow INFO:
	 [Job 97] Completed (qsiprep_wf.single_subject_2001_wf.dwi_preproc_ses_baseline_wf.confounds_wf.fdisp).
220928-08:59:15,360 nipype.workflow INFO:
	 [MultiProc] Running 1 tasks, and 1 jobs ready. Free memory (GB): 23.80/24.00, Free processors: 3/4.
                     Currently running:
                       * qsiprep_wf.single_subject_2001_wf.dwi_preproc_ses_baseline_wf.hmc_sdc_wf.back_to_lps
220928-08:59:15,492 nipype.workflow INFO:
	 [Node] Setting-up "qsiprep_wf.single_subject_2001_wf.dwi_preproc_ses_baseline_wf.confounds_wf.concat" in "/flywheel/v0/work/qsiprep_wf/single_subject_2001_wf/dwi_preproc_ses_baseline_wf/confounds_wf/concat".
220928-08:59:15,505 nipype.workflow INFO:
	 [Node] Executing "concat" <qsiprep.interfaces.confounds.GatherConfounds>
220928-08:59:15,535 nipype.workflow INFO:
	 [Node] Finished "concat", elapsed time 0.028486s.
220928-08:59:15,536 nipype.workflow WARNING:
	 Storing result file without outputs
220928-08:59:15,537 nipype.workflow WARNING:
	 [Node] Error on "qsiprep_wf.single_subject_2001_wf.dwi_preproc_ses_baseline_wf.confounds_wf.concat" (/flywheel/v0/work/qsiprep_wf/single_subject_2001_wf/dwi_preproc_ses_baseline_wf/confounds_wf/concat)
220928-08:59:15,538 nipype.workflow ERROR:
	 Node concat failed to run on host runc.
220928-08:59:15,547 nipype.workflow ERROR:
	 Saving crash info to /flywheel/v0/output/633310d24deeedf8aadfe1aa/qsiprep/sub-2001/log/20220928-042026_82a9f6ee-37b3-4fd4-8fa5-ee15ae3f4f7b/crash-20220928-085915-root-concat-925661be-b07d-48f3-98a6-b33b2c9dd356.txt
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/plugins/multiproc.py", line 344, in _send_procs_to_workers
    self.procs[jobid].run(updatehash=updatehash)
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 524, in run
    result = self._run_interface(execute=True)
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 642, in _run_interface
    return self._run_command(execute)
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 750, in _run_command
    raise NodeExecutionError(
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node concat.

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 398, in run
    runtime = self._run_interface(runtime)
  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/interfaces/confounds.py", line 53, in _run_interface
    combined_out, confounds_list = _gather_confounds(
  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/interfaces/confounds.py", line 178, in _gather_confounds
    raise Exception("Gradients don't match. File a bug report!")
Exception: Gradients don't match. File a bug report!


220928-08:59:31,49 nipype.interface INFO:
	 Re-orienting /flywheel/v0/work/qsiprep_wf/single_subject_2001_wf/dwi_preproc_ses_baseline_wf/hmc_sdc_wf/eddy/eddy_corrected.nii.gz to LPS
220928-08:59:51,17 nipype.interface INFO:
	 Reorienting /flywheel/v0/work/qsiprep_wf/single_subject_2001_wf/dwi_preproc_ses_baseline_wf/hmc_sdc_wf/eddy/eddy_corrected.eddy_rotated_bvecs to LPS
220928-08:59:51,38 nipype.workflow INFO:
	 [Node] Finished "back_to_lps", elapsed time 39.543775s.
[16778553ms   INFO     ]  Command return code: 1
[16778555ms   ERROR    ]  [WARNING] This document format requires a nonempty <title> element.
  Please specify either 'title' or 'pagetitle' in the metadata.
  Falling back to 'CITATION'
 file none does not exist . 
 file none does not exist . 
 file none does not exist . 
QSIPrep failed: Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/plugins/multiproc.py", line 344, in _send_procs_to_workers
    self.procs[jobid].run(updatehash=updatehash)
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 524, in run
    result = self._run_interface(execute=True)
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 642, in _run_interface
    return self._run_command(execute)
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 750, in _run_command
    raise NodeExecutionError(
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node concat.

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 398, in run
    runtime = self._run_interface(runtime)
  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/interfaces/confounds.py", line 53, in _run_interface
    combined_out, confounds_list = _gather_confounds(
  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/interfaces/confounds.py", line 178, in _gather_confounds
    raise Exception("Gradients don't match. File a bug report!")
Exception: Gradients don't match. File a bug report!


Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/plugins/multiproc.py", line 344, in _send_procs_to_workers
    self.procs[jobid].run(updatehash=updatehash)
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 524, in run
    result = self._run_interface(execute=True)
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 642, in _run_interface
    return self._run_command(execute)
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 750, in _run_command
    raise NodeExecutionError(
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node concat.

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 398, in run
    runtime = self._run_interface(runtime)
  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/interfaces/confounds.py", line 53, in _run_interface
    combined_out, confounds_list = _gather_confounds(
  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/interfaces/confounds.py", line 178, in _gather_confounds
    raise Exception("Gradients don't match. File a bug report!")
Exception: Gradients don't match. File a bug report!


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/miniconda/bin/qsiprep", line 8, in <module>
    sys.exit(main())
  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/cli/run.py", line 659, in main
    qsiprep_wf.run(**plugin_settings)
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/workflows.py", line 638, in run
    runner.run(execgraph, updatehash=updatehash, config=self.config)
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/plugins/base.py", line 187, in run
    self._send_procs_to_workers(updatehash=updatehash, graph=graph)
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/plugins/multiproc.py", line 347, in _send_procs_to_workers
    self._clean_queue(
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/plugins/base.py", line 244, in _clean_queue
    raise RuntimeError("".join(result["traceback"]))
RuntimeError: Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/plugins/multiproc.py", line 344, in _send_procs_to_workers
    self.procs[jobid].run(updatehash=updatehash)
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 524, in run
    result = self._run_interface(execute=True)
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 642, in _run_interface
    return self._run_command(execute)
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/pipeline/engine/nodes.py", line 750, in _run_command
    raise NodeExecutionError(
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node concat.

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.8/site-packages/nipype/interfaces/base/core.py", line 398, in run
    runtime = self._run_interface(runtime)
  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/interfaces/confounds.py", line 53, in _run_interface
    combined_out, confounds_list = _gather_confounds(
  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/interfaces/confounds.py", line 178, in _gather_confounds
    raise Exception("Gradients don't match. File a bug report!")
Exception: Gradients don't match. File a bug report!



[16778556ms   CRITICAL ]  The following command has failed: 
['qsiprep', '/flywheel/v0/work/bids', '/flywheel/v0/output/633310d24deeedf8aadfe1aa', 'participant', '-v', '--output-resolution=1', '--n_cpus=4', '--mem_mb=24576', '--participant_label=2001', '--skip-bids-validation']
[16778556ms   ERROR    ]  Unable to execute command.
Traceback (most recent call last):
  File "/flywheel/v0/run.py", line 307, in main
    e_code = run(gear_options, app_options)
  File "/flywheel/v0/fw_gear_bids_qsiprep/main.py", line 135, in run
    exec_command(
  File "/usr/local/miniconda/lib/python3.8/site-packages/flywheel_gear_toolkit/interfaces/command_line.py", line 195, in exec_command
    raise RuntimeError("The following command has failed: \n{}".format(command))
RuntimeError: The following command has failed: 
['qsiprep', '/flywheel/v0/work/bids', '/flywheel/v0/output/633310d24deeedf8aadfe1aa', 'participant', '-v', '--output-resolution=1', '--n_cpus=4', '--mem_mb=24576', '--participant_label=2001', '--skip-bids-validation']
[16778562ms   INFO     ]  Zipping output file bids-qsiprep_ses-baseline_633310d24deeedf8aadfe1aa.zip
[16790446ms   INFO     ]  Creating viewable archives for all html files
[16790446ms   INFO     ]  Found path: /flywheel/v0/output/633310d24deeedf8aadfe1aa/qsiprep
[16790449ms   WARNING  ]  No *.html files at /flywheel/v0/output/633310d24deeedf8aadfe1aa/qsiprep
[16790451ms   INFO     ]  Previous errors:
  Error msg: The following command has failed: 
['qsiprep', '/flywheel/v0/work/bids', '/flywheel/v0/output/633310d24deeedf8aadfe1aa', 'participant', '-v', '--output-resolution=1', '--n_cpus=4', '--mem_mb=24576', '--participant_label=2001', '--skip-bids-validation']

[16790451ms   INFO     ]  flywheel/bids-qsiprep Gear is done.  Returning 1
Gear completed unsuccessfully after running for 4h39m50.983s.
Uploading results...
Found 2 output file(s) for a total size of 288MB.
The following outputs have been saved:
  - bids-qsiprep_ses-baseline_633310d24deeedf8aadfe1aa.zip
  - bids_tree.html

Hello,

What version of qsiprep are you using and what is your command? Are your data BIDS valid?

Best,
Steven

Hi Steven and thanks for your response.

QSIprep version is 1.0.3_0.15.4.

I’m not sure that is bids validated, so I made bids-validation option disabled.
I did it for another non-validated data as well, which was for another scanner type and had a different JSON file(attached).
sub-3005_dir-AP_dwi.txt (3.4 KB)

I think the problem is related to json information.
What do you think?

Hello,

QSIPrep is designed to run on bids datasets. You should only disable this if you are certain your data are bids valid.

What makes you think it is related to the JSON file? Did QSIPrep run for the other dataset? This error seems like a mismatch in number of gradients from the bval/bvec files and number of volumes in the dwi image.

Best,
Steven

Yes, I did for another dataset and that worked well.
That data has a different JSON file, because of that, I said the error may be related to incomplete JSON info. I attach both JSON files.

Also, thanks for your comment on bval/bvec files and image volumes. We should check them again and let you know.

Best,
Majid