[Node] Error on "mriqc_wf.dwiMRIQC.ComputeIQMs.datasink"

Hi all,
I’m new to learn fmriprep relevant knowledge, now i step to MRIQC and encountered some issues, please anyone experts in this community could help me?

:slight_smile:

Summary of what happened:

MRIQC version 24.1.0.dev0+g3fe90466.d20240417 was used to calculate the subject-level analysis, there were 56 subjects in total, and all subject folder generated the JOSN file, but almost half of them no figures and htmls generated. after checked the error, I found the wrong subjects were no info write into the sub-xxx_dwi.json file. I don’t know why this happened, cause all nifti file of each subjects are intact, include the json file. by the way i used heudoconv to convert all DICOM file into NIFTI format.

Command used (and if a helper script was used, a link to the helper script or the command generated):

I installed docker on windows, here is my code

$subids =  $(cat H:\MRI_data\sublist.txt)
foreach($indexs in 0..55)
{
$proc_subs = $subids[$indexs]
docker run -it --rm `
-v F:\attention_data\Nift:/data:ro `
-v F:\attention_data\MRIQC:/out `
nipreps/mriqc:latest /data /out participant --participant_label $proc_subs
}

Version:

MRIQC version 24.1.0.dev0+g3fe90466.d20240417

Environment (Docker, Singularity / Apptainer, custom installation):

docker on windows wsl2

Data formatted according to a validatable standard? Please provide the output of the validator:

passed the online validation

Relevant log outputs (up to 20 lines):

2024-04-25 03:15:14 | INFO     | niworkflows      | Generating visual report
2024-04-25 03:16:16 | WARNING  | nipype.interface | Estimated signal variation in the background was too small (MAD=0.0, sigma=11.2851)
2024-04-25 03:16:16 | WARNING  | nipype.interface | Estimated signal variation in the background was too small (MAD=0.0, sigma=11.2851)
2024-04-25 03:16:16 | WARNING  | nipype.interface | Estimated signal variation in the background was too small (MAD=0.0, sigma=11.2851)
2024-04-25 03:22:26 | WARNING  | nipype.workflow  | Storing result file without outputs
2024-04-25 03:22:26 | WARNING  | nipype.workflow  | [Node] Error on "mriqc_wf.dwiMRIQC.ComputeIQMs.datasink" (/tmp/work/mriqc_wf/dwiMRIQC/ComputeIQMs/_in_file_..data..sub-114..dwi..sub-114_dwi.nii.gz/datasink)

Screenshots / relevant information: Traceback:

    Traceback (most recent call last):
      File "/opt/conda/lib/python3.11/site-packages/nipype/interfaces/base/core.py", line 397, in run
        runtime = self._run_interface(runtime)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/mriqc/interfaces/bids.py", line 188, in _run_interface
        json.dumps(
      File "/opt/conda/lib/python3.11/site-packages/simplejson/__init__.py", line 395, in dumps
        **kw).encode(obj)
              ^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/simplejson/encoder.py", line 300, in encode
        chunks = list(chunks)
                 ^^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/simplejson/encoder.py", line 714, in _iterencode
        for chunk in _iterencode_dict(o, _current_indent_level):
      File "/opt/conda/lib/python3.11/site-packages/simplejson/encoder.py", line 645, in _iterencode_dict
        yield _floatstr(value)
              ^^^^^^^^^^^^^^^^
      File "/opt/conda/lib/python3.11/site-packages/simplejson/encoder.py", line 351, in floatstr
        raise ValueError(
    ValueError: Out of range float values are not JSON compliant: nan

1 Like

This issue seems related to JSON encoders, which are not equipped to parse NaN values. This is not unique to mriqc, but it does disrupt its functionality. The likely cause is that some fields in your JSON file(s) contain NaN values. For additional information, see this discussion on the mriqc-users Google group and this GitHub issue.

In my opinion, the best strategy is to create a small utility script that sanitizes these fields in a JSON-compliant manner, for example by converting NaN values to null, although I haven’t tested this solution yet. Alternatively, these fields could be removed. Regardless of the method chosen, a BIDS validation is recommended after sanitizing the files.

If you decide to write such a utility, please consider sharing it here to assist others facing similar issues. I have encountered this problem with at least two different fMRI datasets in the past few years, and several other users have reported similar errors. While we wait for the mriqc maintainers to resolve this issue, having such a utility to sanitize the NaN JSON fields would be beneficial to the broader community :slight_smile:

1 Like

Just as an update on this, I ended up writing the small Python utility to sanitize NaN values in JSON files, but it turns out this is not what is raising the issue for me.

When I check the crash log, this is what I get:

Node: mriqc_wf.anatMRIQC.ComputeIQMs.datasink
Working directory: /scratch/mriqc_wf/anatMRIQC/ComputeIQMs/_in_file_..data..sub-38..anat..sub-38_space-MNI152NLin2009cAsym_res-2_desc-preproc_T1w.nii.gz/datasink

Node inputs:

_outputs = {'qi_2': 0.0}
acq_id = <undefined>
dataset = <unset>
dismiss_entities = ['part']
in_file = /data/sub-38/anat/sub-38_space-MNI152NLin2009cAsym_res-2_desc-preproc_T1w.nii.gz
metadata = {'Resolution': 'Template MNI152NLin2009cAsym (2.0x2.0x2.0 mm^3), curated by TemplateFlow 23.1.0', 'SkullStripped': False}
modality = T1w
out_dir = /out
provenance = {'md5sum': 'c43e21c9608ccda1d0dd1e32cebffdc5', 'version': '24.1.0.dev0+g3fe90466.d20240417', 'software': 'mriqc', 'settings': {'testing': False}, 'warnings': {'small_air_mask': True, 'large_rot_frame': True}}
rec_id = <undefined>
root = {'summary_csf_mean': 153.9373, 'summary_csf_median': 89.0, 'summary_csf_p95': 454.0, 'summary_csf_p05': 0.0, 'summary_csf_k': -1.3813, 'summary_csf_stdv': 170.1005, 'summary_csf_mad': 131.9516, 'summary_csf_n': 68109.5750447787, 'summary_gm_mean': 550.3031, 'summary_gm_median': 535.0, 'summary_gm_p95': 827.0, 'summary_gm_p05': 297.0, 'summary_gm_k': -0.9245, 'summary_gm_stdv': 169.4326, 'summary_gm_mad': 168.8041, 'summary_gm_n': 104437.57099339308, 'summary_wm_mean': 981.6483, 'summary_wm_median': 994.0, 'summary_wm_p95': 1043.0, 'summary_wm_p05': 871.0, 'summary_wm_k': 0.1604, 'summary_wm_stdv': 51.928, 'summary_wm_mad': 41.109, 'summary_wm_n': 85015.85414913118, 'summary_bg_mean': 0.0, 'summary_bg_median': 0.0, 'summary_bg_p95': 0.0, 'summary_bg_p05': 0.0, **'summary_bg_k': nan**, 'summary_bg_stdv': 0.0, 'summary_bg_mad': 0.0, 'summary_bg_n': 425471.0, 'snr_csf': 0.5232162553235187, 'snr_wm': 19.14177619038615, 'snr_gm': 3.15758265325029, 'snr_total': 7.607525032986653, 'snrd_csf': -1.0, 'snrd_wm': -1.0, 'snrd_gm': -1.0, 'snrd_total': -1.0, 'cnr': 2.5901250537402323, 'fber': -1.0, 'efc': 0.822, 'wm2max': 0.9339389213671632, 'qi_1': 0.0, 'cjv': 0.4573270152505447, 'fwhm_x': 2.77502, 'fwhm_y': 2.928585, 'fwhm_z': 2.826965, 'fwhm_avg': 2.843523333333333, 'icvs_csf': 0.26443850628874715, 'icvs_gm': 0.40548359398455835, 'icvs_wm': 0.3300778997266945, 'rpve_csf': 4.11642406088599, 'rpve_gm': 4.095713126758409, 'rpve_wm': 4.180395826409897, 'size_x': 97, 'size_y': 115, 'size_z': 97, 'spacing_x': 2.0, 'spacing_y': 2.0, 'spacing_z': 2.0, 'inu_range': 0.018933320045471214, 'inu_med': 0.25616756081581116, 'tpm_overlap_csf': 0.25478921608094895, 'tpm_overlap_gm': 0.5756801633183153, 'tpm_overlap_wm': 0.6040912214838626, 'qi_2': 0.0, 'bids_meta': {'subject_id': '38', 'modality': 'T1w', 'Resolution': 'Template MNI152NLin2009cAsym (2.0x2.0x2.0 mm^3), curated by TemplateFlow 23.1.0', 'SkullStripped': False, 'dataset': '<unset>'}, 'provenance': {'md5sum': 'c43e21c9608ccda1d0dd1e32cebffdc5', 'version': '24.1.0.dev0+g3fe90466.d20240417', 'software': 'mriqc', 'settings': {'testing': False}, 'warnings': {'small_air_mask': True, 'large_rot_frame': True}}}
run_id = <undefined>
session_id = <undefined>
subject_id = 38
task_id = <undefined>

Traceback (most recent call last):
  File "/opt/conda/lib/python3.11/site-packages/nipype/pipeline/plugins/linear.py", line 47, in run
    node.run(updatehash=updatehash)
  File "/opt/conda/lib/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node datasink.

Traceback:
	Traceback (most recent call last):
	  File "/opt/conda/lib/python3.11/site-packages/nipype/interfaces/base/core.py", line 397, in run
	    runtime = self._run_interface(runtime)
	              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
	  File "/opt/conda/lib/python3.11/site-packages/mriqc/interfaces/bids.py", line 188, in _run_interface
	    json.dumps(
	  File "/opt/conda/lib/python3.11/site-packages/simplejson/__init__.py", line 395, in dumps
	    **kw).encode(obj)
	          ^^^^^^^^^^^
	  File "/opt/conda/lib/python3.11/site-packages/simplejson/encoder.py", line 300, in encode
	    chunks = list(chunks)
	             ^^^^^^^^^^^^
	  File "/opt/conda/lib/python3.11/site-packages/simplejson/encoder.py", line 714, in _iterencode
	    for chunk in _iterencode_dict(o, _current_indent_level):
	  File "/opt/conda/lib/python3.11/site-packages/simplejson/encoder.py", line 645, in _iterencode_dict
	    yield _floatstr(value)
	          ^^^^^^^^^^^^^^^^
	  File "/opt/conda/lib/python3.11/site-packages/simplejson/encoder.py", line 351, in floatstr
	    raise ValueError(
	ValueError: Out of range float values are not JSON compliant: nan

Apparently the problem is not with opening, but rather with saving a JSON file containing NaNs. The offending field seems to be 'summary_bg_k': nan.

Any idea from where this nan comes from? I am running mriqc on a dataset preprocessed with fMRIprep – could this be the cause?

Pinging @oesteban in case he knows more.

Thanks!

Please follow up in the repo - someone posted the same error there: DWI bids compliant but MRIQC cannot process · Issue #1302 · nipreps/mriqc · GitHub

We’ll be discussing this there :).

I checked the discussion, but it seems focused on the DWI images (that I don’t have), and does not provide any hints to a solution.

@oesteban Any idea of what could be causing the ‘summary_bg_k’: nan issue? I’d like to use mriqc and include it in my manuscript, but this has been a significant bottleneck so far.

Thanks!

Hi,

I think other way is to do through correctly formatted NIfTI files. So you need to make sure that all your NIfTI files are intact and correctly formatted. You can use tools like nibabel in Python to verify the integrity of your NIfTI files.

import nibabel as nib

nifti_file = nib.load('path_to_your_nifti_file.nii.gz')
print(nifti_file)

Thanks
Oliva :slightly_smiling_face:

Hi @iamolivasmith,

I am not sure I understand your suggestion. The images I am using are the output of fmriprep, and I can open and visualize them without any issues with other tools. Also, I have this issue with basically all my subjects.

It would be nice to have at least a few pointers on how to debug this, and how the summary_bg_k is populated.

Best,
Andrea

I am talking about the tool nibabel, well let me check this again.

Thanks