Extract_deltam failed to run

cindylucerog · November 23, 2024, 11:37pm

Summary of what happened:

Hi everyone,
I saw someone posted a similar error in the past, but it is not clear if it was solved or if there is a workaround. I get the error Node extract_deltam failed to run on host followed by nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node extract_deltam. Finally, I see a message ValueError: PostLabelingDelay is an array, but the number of values (90) does not match the number of volumes in the ASL data (88).

I think is coming from the previously mentioned “dummy” scans, or “noise” scans. In the case of the HCP-Aging dataset, ASL sequence has 43 pairs of label and control, followed by two “noise” scans and two M0 images, for a total of 90 images.

BIDS provides several ASL examples and among them they have some coming from this same dataset, so I made sure I was following that and that all parameters indeed made sense. See their example here.

I think the next step would be to split the images into ASL images and M0 files; before I do that, do you know if there is a known solution/workaround? I saw a post where someone tried to split everything and still got errors. Your help is very much appreciated

ASL JSON file example:

{
    "Modality": "MR",
    "MagneticFieldStrength": 3,
    "Manufacturer": "Siemens",
    "ManufacturersModelName": "Prisma",
    "InstitutionName": "CMRR_3T-C",
    "InstitutionalDepartmentName": "Department",
    "InstitutionAddress": "MN_US",
    "DeviceSerialNumber": "166XX",
    "StationName": "TRIOC",
    "BodyPartExamined": "BRAIN",
    "PatientPosition": "HFS",
    "ProcedureStepDescription": "HCP_PRIVATE_HCP",
    "SoftwareVersions": "syngo_MR_E11",
    "MRAcquisitionType": "2D",
    "SeriesDescription": "mbPCASLhr_PA",
    "ProtocolName": "mbPCASLhr_PA",
    "ScanningSequence": "EP",
    "SequenceVariant": "SK",
    "ScanOptions": "PFP_FS",
    "SequenceName": "mbPCASL2d1_86",
    "ImageType": [
        "ORIGINAL",
        "PRIMARY",
        "M",
        "ND",
        "MOSAIC"
    ],
    "SeriesNumber": 31,
    "AcquisitionTime": "10:28:38.982500",
    "AcquisitionNumber": 1,
    "SliceThickness": 2.27,
    "SpacingBetweenSlices": 2.497,
    "SAR": 0.376648,
    "EchoTime": 0.019,
    "RepetitionTime": 3.58,
    "FlipAngle": 90,
    "PartialFourier": 0.75,
    "BaseResolution": 86,
    "ShimSetting": [
        -900,
        -11564,
        -3916,
        175,
        -105,
        -105,
        215,
        -21
    ],
    "TxRefAmp": 279.018,
    "PhaseResolution": 1,
    "ReceiveCoilName": "Head_32",
    "ReceiveCoilActiveElements": "HEA;HEP",
    "PulseSequenceDetails": "%CustomerSeq%_XFL_VE11C_ep2d_mbpcasl_HCP",
    "ConsistencyInfo": "N4_VE11C_LATEST_20160120",
    "MultibandAccelerationFactor": 6,
    "PercentPhaseFOV": 100,
    "EchoTrainLength": 65,
    "PhaseEncodingSteps": 65,
    "AcquisitionMatrixPE": 86,
    "ReconMatrixPE": 86,
    "BandwidthPerPixelPhaseEncode": 20.4,
    "EffectiveEchoSpacing": 0.000569995,
    "DerivedVendorReportedEchoSpacing": 0.000569995,
    "TotalReadoutTime": 0.0484496,
    "PixelBandwidth": 2325,
    "DwellTime": 2.5e-06,
    "PhaseEncodingDirection": "j",
    "SliceTiming": [
        0,
        0.06,
        0.1175,
        0.1775,
        0.2375,
        0.295,
        0.355,
        0.4125,
        0.4725,
        0.5325,
        0,
        0.06,
        0.1175,
        0.1775,
        0.2375,
        0.295,
        0.355,
        0.4125,
        0.4725,
        0.5325,
        0,
        0.06,
        0.1175,
        0.1775,
        0.2375,
        0.295,
        0.355,
        0.4125,
        0.4725,
        0.5325,
        0,
        0.06,
        0.1175,
        0.1775,
        0.2375,
        0.295,
        0.355,
        0.4125,
        0.4725,
        0.5325,
        0,
        0.06,
        0.1175,
        0.1775,
        0.2375,
        0.295,
        0.355,
        0.4125,
        0.4725,
        0.5325,
        0,
        0.06,
        0.1175,
        0.1775,
        0.2375,
        0.295,
        0.355,
        0.4125,
        0.4725,
        0.5325
    ],
    "ImageOrientationPatientDICOM": [
        0.998816,
        0.0459492,
        0.0159594,
        -0.0420375,
        0.980486,
        -0.19204
    ],
    "InPlanePhaseEncodingDirectionDICOM": "COL",
    "ConversionSoftware": "dcm2niix",
    "ConversionSoftwareVersion": "v1.0.20171017 GCC4.4.7",
    "M0Type": "Included",
    "ArterialSpinLabelingType": "PCASL",
    "PostLabelingDelay": [
        0.2,
        0.2,
        0.2,
        0.2,
        0.2,
        0.2,
        0.2,
        0.2,
        0.2,
        0.2,
        0.2,
        0.2,
        0.7,
        0.7,
        0.7,
        0.7,
        0.7,
        0.7,
        0.7,
        0.7,
        0.7,
        0.7,
        0.7,
        0.7,
        1.2,
        1.2,
        1.2,
        1.2,
        1.2,
        1.2,
        1.2,
        1.2,
        1.2,
        1.2,
        1.2,
        1.2,
        1.7,
        1.7,
        1.7,
        1.7,
        1.7,
        1.7,
        1.7,
        1.7,
        1.7,
        1.7,
        1.7,
        1.7,
        1.7,
        1.7,
        1.7,
        1.7,
        1.7,
        1.7,
        1.7,
        1.7,
        2.2,
        2.2,
        2.2,
        2.2,
        2.2,
        2.2,
        2.2,
        2.2,
        2.2,
        2.2,
        2.2,
        2.2,
        2.2,
        2.2,
        2.2,
        2.2,
        2.2,
        2.2,
        2.2,
        2.2,
        2.2,
        2.2,
        2.2,
        2.2,
        2.2,
        2.2,
        2.2,
        2.2,
        2.2,
        2.2,
        0.0,
        0.0,
        0.0,
        0.0
    ],
    "AcquisitionVoxelSize": [
        2.5,
        2.5,
        2.497
    ],
    "BackgroundSuppression": false,
    "TotalAcquiredPairs": 43,
    "LabelingDuration": [
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        1.5,
        0,
        0,
        0,
        0
    ],
    "RepetitionTimePreparation": [
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        3.58,
        8.0,
        8.0
    ]
}

aslcontext.tsv:


volume_type
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
label
control
noRF
noRF
m0scan
m0scan

Command used (and if a helper script was used, a link to the helper script or the command generated):

export FS_LICENSE=/home/m/mchakrav/cindylu/scratch/analysis/ASLprep/freesurferlicense/license.txt
export TEMPLATEFLOW_HOME=/home/m/mchakrav/cindylu/scratch/analysis/templateflow
export APPTAINERENV_FS_LICENSE=/freesurfer_license/license.txt
export APPTAINERENV_TEMPLATEFLOW_HOME=/templateflow


singularity run --cleanenv \
  --env TEMPLATEFLOW_HOME=/templateflow \
  --bind /home/m/mchakrav/cindylu/scratch/analysis/ASLprep/data3subjects:/data:ro \
  --bind /home/m/mchakrav/cindylu/scratch/analysis/ASLprep/output3subjects:/out \
  --bind /home/m/mchakrav/cindylu/scratch/analysis/ASLprep/intermediatefiles:/work \
  --bind /home/m/mchakrav/cindylu/scratch/analysis/ASLprep/freesurferlicense:/freesurfer_license \
  --bind /home/m/mchakrav/cindylu/scratch/analysis/templateflow:/templateflow \
  /home/m/mchakrav/cindylu/scratch/analysis/ASLprep/aslprep-0.7.4.simg \
  /data /out participant \
  --skip_bids_validation \
  --output-spaces T1w \
  --basil \
  --fs-license-file /freesurfer_license/license.txt \
  --work-dir /work

Version:

aslprep-0.7.4

Environment (Docker, Singularity / Apptainer, custom installation):

Singularity

Data formatted according to a validatable standard? Please provide the output of the validator:

Find below warnings from BIDS Validator, could not paste the BIDS validator JSON log due to its length.


warning: UNKNOWN_BIDS_VERSION

warning: README_FILE_MISSING

warning: TOO_FEW_AUTHORS

warning: JSON_KEY_RECOMMENDED (License)

warning: JSON_KEY_RECOMMENDED (GeneratedBy)

warning: JSON_KEY_RECOMMENDED (SourceDatasets)

warning: NO_AUTHORS (Authors)

warning: SIDECAR_KEY_RECOMMENDED (GradientSetType)

warning: SIDECAR_KEY_RECOMMENDED (MRTransmitCoilSequence)

warning: SIDECAR_KEY_RECOMMENDED (MatrixCoilMode)

warning: SIDECAR_KEY_RECOMMENDED (CoilCombinationMethod)

warning: SIDECAR_KEY_RECOMMENDED (PulseSequenceType)

warning: SIDECAR_KEY_RECOMMENDED (MTState)

warning: SIDECAR_KEY_RECOMMENDED (SpoilingState)

warning: SIDECAR_KEY_RECOMMENDED (NumberShots)

warning: SIDECAR_KEY_RECOMMENDED (ParallelReductionFactorOutOfPlane)

warning: SIDECAR_KEY_RECOMMENDED (ParallelAcquisitionTechnique)

warning: SIDECAR_KEY_RECOMMENDED (PartialFourierDirection)

warning: SIDECAR_KEY_RECOMMENDED (EffectiveEchoSpacing)

warning: SIDECAR_KEY_RECOMMENDED (MixingTime)

warning: SIDECAR_KEY_RECOMMENDED (PhaseEncodingDirection)

warning: SIDECAR_KEY_RECOMMENDED (TotalReadoutTime)

warning: SIDECAR_KEY_RECOMMENDED (EchoTime)

warning: SIDECAR_KEY_RECOMMENDED (MultibandAccelerationFactor)

warning: SIDECAR_KEY_RECOMMENDED (B0FieldIdentifier)

warning: SIDECAR_KEY_RECOMMENDED (ParallelReductionFactorInPlane)

warning: SIDECAR_KEY_RECOMMENDED (InversionTime)

warning: SIDECAR_KEY_RECOMMENDED (SliceEncodingDirection)

warning: SIDECAR_KEY_RECOMMENDED (VascularCrushing)

warning: SIDECAR_KEY_RECOMMENDED (LabelingOrientation)

warning: SIDECAR_KEY_RECOMMENDED (LabelingDistance)

warning: SIDECAR_KEY_RECOMMENDED (LabelingLocationDescription)

warning: SIDECAR_KEY_RECOMMENDED (LabelingPulseAverageGradient)

warning: SIDECAR_KEY_RECOMMENDED (LabelingPulseMaximumGradient)

warning: SIDECAR_KEY_RECOMMENDED (LabelingPulseAverageB1)

warning: SIDECAR_KEY_RECOMMENDED (LabelingPulseDuration)

warning: SIDECAR_KEY_RECOMMENDED (LabelingPulseFlipAngle)

warning: SIDECAR_KEY_RECOMMENDED (LabelingPulseInterval)

warning: SIDECAR_KEY_RECOMMENDED (PCASLType)

warning: GZIP_HEADER_MTIME

warning: TOTAL_ACQUIRED_VOLUMES_NOT_CONSISTENT

warning: POST_LABELING_DELAY_GREATER

warning: LABELING_DURATION_GREATER

Relevant log outputs (up to 20 lines):

41123-13:02:09,984 nipype.workflow WARNING:
	 [Node] Error on "aslprep_0_7_wf.sub_HCA6002236_wf.asl_preproc_dir_PA_wf.cbf_wf.extract_deltam" (/work/aslprep_0_7_wf/sub_HCA6002236_wf/asl_preproc_dir_PA_wf/cbf_wf/extract_deltam)
241123-13:02:09,988 nipype.workflow ERROR:
	 Node extract_deltam failed to run on host nia1721.scinet.local.
241123-13:02:09,994 nipype.workflow ERROR:
	 Saving crash info to /out/sub-HCA6002236/log/20241123-085344_a4b820cb-9e37-4a8e-b1c4-ec6ce122635e/crash-20241123-130209-cindylu-extract_deltam-f4c06185-1b33-4c22-8585-1aec9698b431.txt
Traceback (most recent call last):
  File "/opt/conda/envs/aslprep/lib/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 344, in _send_procs_to_workers
    self.procs[jobid].run(updatehash=updatehash)
  File "/opt/conda/envs/aslprep/lib/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/aslprep/lib/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/aslprep/lib/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node extract_deltam.

Traceback:
	Traceback (most recent call last):
	  File "/opt/conda/envs/aslprep/lib/python3.11/site-packages/nipype/interfaces/base/core.py", line 397, in run
	    runtime = self._run_interface(runtime)
	              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
	  File "/opt/conda/envs/aslprep/lib/python3.11/site-packages/aslprep/interfaces/cbf.py", line 273, in _run_interface
	    raise ValueError(
	ValueError: PostLabelingDelay is an array, but the number of values (90) does not match the number of volumes in the ASL data (88).

Screenshots / relevant information:

Steven · November 23, 2024, 11:58pm

Hi @cindylucerog,

Please fill out the software support post template, which I re-added back in to your post.

Best,
Steven

tsalo · November 25, 2024, 5:10pm

“Dummy” scans and “noise” scans aren’t the same thing. “Dummy” scans refers to volumes at the beginning of an acquisition where there might be artifacts or heightened contrast. The noise scans, assuming the HCP-Aging protocol is actually using noRF scans, are useful for estimating the level of thermal noise in the scan. They can be used with NORDIC, for example, to remove thermal noise from the other volumes (not that I’ve ever done that with ASL data).

ASLPrep includes a --dummy-scans parameter to match fMRIPrep, but I’ve never actually used it and I don’t know if it works as intended.

It would be really helpful if you filled in the remaining fields in the Software Support template as @Steven suggested. If you used --dummy-scans with the intention of flagging these noRF volumes, that might be the culprit.

cindylucerog · November 25, 2024, 6:46pm

Hello Steven and Taylor,

Sorry for the delay; I was traveling to my home country. I have updated my first post to provide the information.

Thank you both,

-Cindy

tsalo · November 26, 2024, 1:38pm

Thank you @cindylucerog.

I have managed to reproduce the bug you’ve identified and I’ll work on it today. Hopefully I’ll be able to push a fix by EOD, but if not I’ll be able to circle back to it on Monday.

cindylucerog · November 26, 2024, 2:11pm

Thank you very much for your help in this matter. I appreciate your help so much and I’ll be alert for any changes,

-Cindy

tsalo · November 27, 2024, 2:12pm

@cindylucerog can you try the new unstable version on DockerHub? If that works, I’ll make a new release.

cindylucerog · November 29, 2024, 1:36am

Hi @tsalo,

I was able to run the unstable version. I got a new error, see below:

241128-20:10:29,832 nipype.workflow WARNING:
	 [Node] Error on "aslprep_0_7_wf.sub_HCA6010538_wf.asl_preproc_dir_PA_wf.cbf_wf.extract_deltam" (/work/aslprep_0_7_wf/sub_HCA6010538_wf/asl_preproc_dir_PA_wf/cbf_wf/extract_deltam)
241128-20:10:29,834 nipype.workflow ERROR:
	 Node extract_deltam failed to run on host nia1913.scinet.local.
241128-20:10:29,839 nipype.workflow ERROR:
	 Saving crash info to /out/sub-HCA6010538/log/20241128-152206_c03ce490-bae6-4684-8e03-04c7cb516bac/crash-20241128-201029-cindylu-extract_deltam-0e14554e-0978-484b-86f5-a3e4305400d1.txt
Traceback (most recent call last):
  File "/opt/conda/envs/aslprep/lib/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 344, in _send_procs_to_workers
    self.procs[jobid].run(updatehash=updatehash)
  File "/opt/conda/envs/aslprep/lib/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/aslprep/lib/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/aslprep/lib/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node extract_deltam.

Traceback:
	Traceback (most recent call last):
	  File "/opt/conda/envs/aslprep/lib/python3.11/site-packages/nipype/interfaces/base/core.py", line 400, in run
	    outputs = self.aggregate_outputs(runtime)
	              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
	  File "/opt/conda/envs/aslprep/lib/python3.11/site-packages/nipype/interfaces/base/core.py", line 461, in aggregate_outputs
	    raise error
	  File "/opt/conda/envs/aslprep/lib/python3.11/site-packages/nipype/interfaces/base/core.py", line 453, in aggregate_outputs
	    setattr(outputs, key, val)
	  File "/opt/conda/envs/aslprep/lib/python3.11/site-packages/traits/base_trait_handler.py", line 74, in error
	    raise TraitError(
	traits.trait_errors.TraitError: The 'm0tr' trait of a _ExtractCBFOutputSpec instance must be a float or None, but a value of array([8., 8.]) <class 'numpy.ndarray'> was specified.

Thank you again for your time investigating this errors,

-Cindy

tsalo · December 2, 2024, 3:01pm

Oof, I think I might know what’s going on there. I’ll try to reproduce it on my end but I think it’ll be a couple of days before I can push a fix.

cindylucerog · December 6, 2024, 8:08am

@tsalo

Thank you for your help, I am alert for any changes,

Best regards,

-Cindy

tsalo · December 9, 2024, 1:59pm

@cindylucerog can you try the 0.7.5rc2 release?

cindylucerog · December 11, 2024, 8:16pm

Hi @tsalo,

Thank you for working on this. It worked, it took 9.5h to run for two subjects and I don’t know how accurate the results below are, I still feel is great progress, thanks to your help this new data might be processed with ASLprep.

I’ll open new requests for support, but this is fixed I believe.

Thank you!