Heudiconv converted func DICOMS into 3d nii instead of 4d

Summary of what happened:

Hi all,

I checked the ‘.heudiconv’ folder, and ‘dicominfo.tsv’ showed my resting-state func scans have different ‘series_id’ (e.g., ‘8001-ep2d_bold_p2’,‘8002-ep2d_bold_p2’,…,‘8240-ep2d_bold_p2’)

Could I stack these 3d nii into 4d by Heudiconv?

Much appreciated

Command used (and if a helper script was used, a link to the helper script or the command generated):

  • Command used in conda
heudiconv -d ./sourcedata/Site_CD5120/SS01/{subject}/DICOM/*/*/* -o ./rawdata/CD5120/SS01 -f ./sourcedata/heuristic_cd5120.py -s 009 010 011 012 013 014 015 016 017 018 -c dcm2niix -b --overwrite
  • ‘heuristic_cd5120.py’
import os
def create_key(template, outtype=('nii.gz',), annotation_classes=None):
    if template is None or not template:
        raise ValueError('Template must be a valid format string')
    return template, outtype, annotation_classes
def infotodict(seqinfo):
    func = create_key('sub-{subject}/func/sub-{subject}_func_cd5120')
    info = {anat: [], func: [], dti: []}
    last_run = len(seqinfo)
    for idx, s in enumerate(seqinfo):
        if ('ep2d_bold' in s.protocol_name):
            info[func].append(s.series_id)
    return info
  • ‘009.auto.txt’/‘009.edit.txt’ in ‘.heudiconv’:
{('sub-{subject}/func/sub-{subject}_func_cd5120', ('nii.gz',), None): ['8001-ep2d_bold_p2',
                                                                       '8002-ep2d_bold_p2',
                                                                       ...
                                                                       '8240-ep2d_bold_p2']}
  • ‘filegroup.json’ in ‘.heudiconv’:
{
  "8001-ep2d_bold_p2": [
    "./sourcedata/Site_CD5120/SS01/009/DICOM/21070613/05460000/63545655",
    "./sourcedata/Site_CD5120/SS01/009/DICOM/21070613/05460000/63545666",
    ...
    "./sourcedata/Site_CD5120/SS01/009/DICOM/21070613/05460000/63546634"
  ],
  ...
  "8240-ep2d_bold_p2": [
    "./sourcedata/Site_CD5120/SS01/009/DICOM/21070613/05460015/63624525",
    ...
    "./sourcedata/Site_CD5120/SS01/009/DICOM/21070613/05460015/63624844"
  ]
}

Version:

heudiconv version 0.13.1
dcm2niix version v1.0.20230411

Environment (Docker, Singularity, custom installation):

miniconda3 environment

Data format:

├── 009
│   └── DICOM
│       ├── 21070613
│       │   ├── 05460000
│       │   │   ├── 63541684
│       │   │   └── 63541695
│       │   │   └── ...
│       │   ├── 05460001
│       │   │   ├── 63547184
│       │   │   ├── 63547195
│       │   │   └── ...

Relevant log outputs:

INFO: Post-treating ./rawdata/CD5120/SS01/sub-009/func/sub-009_func_cd5120.json file
INFO: Converting ./rawdata/CD5120/SS01/sub-009/func/sub-009_func_cd5120 (30 DICOMs) -> ./rawdata/CD5120/SS01/sub-009/func . Converter: dcm2niix . Output types: ('nii.gz',)
230917-09:53:09,819 nipype.workflow INFO:
         [Node] Setting-up "convert" in "/tmp/dcm2niix060ve0cm/convert".
INFO: [Node] Setting-up "convert" in "/tmp/dcm2niix060ve0cm/convert".
230917-09:53:09,843 nipype.workflow INFO:
         [Node] Executing "convert" <nipype.interfaces.dcm2nii.Dcm2niix>
INFO: [Node] Executing "convert" <nipype.interfaces.dcm2nii.Dcm2niix>
230917-09:53:10,126 nipype.interface INFO:
         stdout 2023-09-17T09:53:10.126776:Chris Rorden's dcm2niiX version v1.0.20230411  GCC12.2.0 x86-64 (64-bit Linux)
INFO: stdout 2023-09-17T09:53:10.126776:Chris Rorden's dcm2niiX version v1.0.20230411  GCC12.2.0 x86-64 (64-bit Linux)
230917-09:53:10,127 nipype.interface INFO:
         stdout 2023-09-17T09:53:10.126776:Found 30 DICOM file(s)
INFO: stdout 2023-09-17T09:53:10.126776:Found 30 DICOM file(s)
230917-09:53:10,127 nipype.interface INFO:
         stdout 2023-09-17T09:53:10.126776:Warning: Assuming mosaics saved in reverse order due to 'sSliceArray.ucImageNumb'
INFO: stdout 2023-09-17T09:53:10.126776:Warning: Assuming mosaics saved in reverse order due to 'sSliceArray.ucImageNumb'
230917-09:53:10,127 nipype.interface INFO:
         stdout 2023-09-17T09:53:10.126776:Warning: Siemens XA exported as classic not enhanced DICOM (issue 236)
INFO: stdout 2023-09-17T09:53:10.126776:Warning: Siemens XA exported as classic not enhanced DICOM (issue 236)
230917-09:53:10,127 nipype.interface INFO:
         stdout 2023-09-17T09:53:10.126776:Convert 30 DICOM as ./rawdata/CD5120/SS01/sub-009/func/sub-009_func_cd5120_heudiconv063 (64x64x30x1)
INFO: stdout 2023-09-17T09:53:10.126776:Convert 30 DICOM as ./rawdata/CD5120/SS01/sub-009/func/sub-009_func_cd5120_heudiconv063 (64x64x30x1)
230917-09:53:10,235 nipype.interface INFO:
         stdout 2023-09-17T09:53:10.235645:Conversion required 0.351797 seconds (0.141253 for core code).
INFO: stdout 2023-09-17T09:53:10.235645:Conversion required 0.351797 seconds (0.141253 for core code).
230917-09:53:10,300 nipype.workflow INFO:
         [Node] Finished "convert", elapsed time 0.455839s.
INFO: [Node] Finished "convert", elapsed time 0.455839s.
230917-09:53:10,394 nipype.workflow INFO:
         [Node] Setting-up "embedder" in "/tmp/embedmetax0ey3pgs/embedder".
INFO: [Node] Setting-up "embedder" in "/tmp/embedmetax0ey3pgs/embedder".
230917-09:53:10,412 nipype.workflow INFO:
         [Node] Executing "embedder" <nipype.interfaces.utility.wrappers.Function>
INFO: [Node] Executing "embedder" <nipype.interfaces.utility.wrappers.Function>
230917-09:53:11,244 nipype.workflow INFO:
         [Node] Finished "embedder", elapsed time 0.828648s.
INFO: [Node] Finished "embedder", elapsed time 0.828648s.
230917-09:53:11,244 nipype.workflow WARNING:
         Storing result file without outputs
WARNING: Storing result file without outputs
230917-09:53:11,245 nipype.workflow WARNING:
         [Node] Error on "embedder" (/tmp/embedmetax0ey3pgs/embedder)
WARNING: [Node] Error on "embedder" (/tmp/embedmetax0ey3pgs/embedder)
ERROR: Embedding failed: Exception raised while executing Node embedder.

Traceback:
        Traceback (most recent call last):
          File "./miniconda3/envs/heudiconv/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 397, in run
            runtime = self._run_interface(runtime)
          File "./miniconda3/envs/heudiconv/lib/python3.10/site-packages/nipype/interfaces/utility/wrappers.py", line 142, in _run_interface
            out = function_handle(**args)
          File "<string>", line 35, in embed_dicom_and_nifti_metadata
          File "./miniconda3/envs/heudiconv/lib/python3.10/site-packages/dcmstack/dcmstack.py", line 1226, in parse_and_stack
            results[key] = stack_group(group, warn_on_except, **stack_args)
          File "./miniconda3/envs/heudiconv/lib/python3.10/site-packages/dcmstack/dcmstack.py", line 1180, in stack_group
            result.add_dcm(dcm, meta)
          File "./miniconda3/envs/heudiconv/lib/python3.10/site-packages/dcmstack/dcmstack.py", line 605, in add_dcm
            nii_wrp = NiftiWrapper.from_dicom_wrapper(dw, meta)
          File "./miniconda3/envs/heudiconv/lib/python3.10/site-packages/dcmstack/dcmmeta.py", line 1536, in from_dicom_wrapper
            affine = np.dot(np.diag([-1., -1., 1., 1.]), dcm_wrp.get_affine())
        AttributeError: 'Wrapper' object has no attribute 'get_affine'

Screenshots / relevant information:

This question is underspecified. Can you provide the text from one of the json files created by dcm2niix? I am going to speculate the this data either came from:

  1. Philips scanner where the user selected Series Split. Solution: export data from console without explicitly requiring series split.
  2. early Siemens XA scanner where the user selected either classic or mosaic output. Solution: export data from console as enhanced DICOM.

Edit: looking at the errors, my assumption was correct. dcm2niix is correctly telling you this data comes from a Siemens XA exported as classic data. You will want to export the data as enhanced DICOM. Beyond the series information, the classic data are impoverished (a bit less with recent versions) and Siemens considers the mosaic for visual inspection only and not for subsequent analyses (hence they are exceptionally impoverished). This reflects a limitation of your images, not dcm2niix. The Siemens Research Collaboration Manager associated with your center can provide more details.

Thank you for your reply!
Here is the JSON file created by dcm2niix, your assumption is correct.

{
  "AcquisitionMatrixPE": 64,
  "AcquisitionNumber": 1,
  "BandwidthPerPixelPhaseEncode": 57.87,
  "BaseResolution": 64,
  "BodyPartExamined": "BRAIN",
  "CoilCombinationMethod": "Sum of Squares",
  "CoilString": "HeadNeck_20_TCS",
  "ConversionSoftware": "dcm2niix",
  "ConversionSoftwareVersion": "v1.0.20230411",
  "DerivedVendorReportedEchoSpacing": 0.000540003,
  "DeviceSerialNumber": "176160",
  "DwellTime": 3.2e-06,
  "EchoTime": 0.03,
  "EchoTrainLength": 31,
  "EffectiveEchoSpacing": 0.000270002,
  "FlipAngle": 90,
  "HeudiconvVersion": "0.13.1",
  "ImageOrientationPatientDICOM": [0.992337, 0.109463, -0.057321, -0.11868, 0.97348, -0.195581],
  "ImageType": [
    "ORIGINAL",
    "PRIMARY",
    "M",
    "DIS2D",
    "MFSPLIT"
],
  "ImageTypeText": [
    "ORIGINAL",
    "PRIMARY",
    "M",
    "NONE"
],
  "ImagingFrequency": 123.259997,
  "InPlanePhaseEncodingDirectionDICOM": "COL",
  "MRAcquisitionType": "2D",
  "MagneticFieldStrength": 3,
  "Manufacturer": "Siemens",
  "ManufacturersModelName": "MAGNETOM Vida",
  "MatrixCoilMode": "GRAPPA",
  "Modality": "MR",
  "NonlinearGradientCorrection": true,
  "ParallelReductionFactorInPlane": 2,
  "PartialFourier": 1,
  "PatientPosition": "HFS",
  "PercentPhaseFOV": 100,
  "PercentSampling": 100,
  "PhaseEncodingDirection": "j-",
  "PhaseEncodingSteps": 64,
  "PhaseResolution": 1,
  "PixelBandwidth": 2441,
  "ProcedureStepDescription": "HEAD NECK Head",
  "ProtocolName": "ep2d_bold_p2",
  "PulseSequenceDetails": "%SiemensSeq%\\ep2d_bold",
  "ReceiveCoilActiveElements": "HE1-4",
  "ReceiveCoilName": "HeadNeck_20_TCS",
  "ReconMatrixPE": 64,
  "RefLinesPE": 30,
  "RepetitionTime": 2,
  "SAR": 0.169049,
  "ScanOptions": "FS\\PER",
  "ScanningSequence": "GR\\EP",
  "SequenceName": "*epfid2d1_64",
  "SequenceVariant": "SK",
  "SeriesDescription": "ep2d_bold_p2",
  "SeriesNumber": 8001,
  "ShimSetting": [-1032, 8952, 9527, 433, -46, -137, -112, 46],
  "SliceThickness": 5,
  "SliceTiming": [0.99, 0, 1.055, 0.065, 1.1225, 0.1325, 1.1875, 0.1975, 1.2525, 0.265, 1.32, 0.33, 1.385, 0.395, 1.4525, 0.4625, 1.5175, 0.5275, 1.5825, 0.595, 1.65, 0.66, 1.715, 0.725, 1.7825, 0.7925, 1.8475, 0.8575, 1.9125, 0.925],
  "SoftwareVersions": "syngo MR XA10",
  "SpacingBetweenSlices": 5,
  "TotalReadoutTime": 0.0170101,
  "TxRefAmp": 415.485,
  "VendorReportedEchoSpacing": 0.00054}

OK, I am going to assume (hope) this is archival data and this system has been upgraded to a more recent version (XA20 or later). This will make exporting the data as enhanced DICOMs impossible.

My sense is that the easiest approach now is to use fslmerge to concatenate your NIfTI images. The other option would be to use dcmodify to correct the series number.

Thanks for the suggestion!
Unfortunately, this data comes from one of our sub-centers. :sob:

Anyway, I’ve written a script hoping it will help people struggling with the same thing I am.

import os
import glob
import nibabel
from tqdm import tqdm
import shutil

# input bids directory
bids_directory_path = input("Enter the path to the BIDS directory: ")

# input handler
while not os.path.exists(bids_directory_path):
    print("Error: the path does not exist.")
    bids_directory_path = input("Enter the path to the BIDS directory: ")

# for every subject folder in the bids directory
for subject in tqdm(os.listdir(bids_directory_path)):
    # subject folder is starting with sub-
    if subject.startswith("sub-"):
        # if func folder contains more than 10 files
        if len(os.listdir(os.path.join(bids_directory_path, subject, "func"))) > 10:
            # backup the func folder
            # if func_backup folder exists, delete it
            if os.path.exists(os.path.join(bids_directory_path, subject, "func_backup")):
                shutil.rmtree(os.path.join(
                    bids_directory_path, subject, "func_backup"))
            shutil.copytree(os.path.join(bids_directory_path, subject, "func"),
                            os.path.join(bids_directory_path, subject, "func_backup"))
            # function: nibabel.funcs.concat_images(images)
            # input: sequence of *.nii.gz in the func folder
            concatenated_images = nibabel.funcs.concat_images(
                sorted(glob.glob(os.path.join(bids_directory_path, subject, "func", "*.nii.gz"))))
            # save the concatenated image
            nibabel.save(concatenated_images, os.path.join(bids_directory_path, subject, "func",
                                                      subject + "_concatenated_bold.nii.gz"))

Again, thanks to all of you.