Hi @gavkhar_a,
It looks like both your DWI and fmaps have the PA phase encoding direction, in which case they cannot be used to correct each other. Can you confirm the phase encoding directiosn by looking at the associated jsons? Also your bval and bvecs have a different name (missing the dir-<> label).
When fixing those errors please run the BIDS validator on the dataset to find potential other errors with data/metadata organization.
Best,
Steven
Thanks so much Steven!
Yes, since the fmaps were named as “topup” runs, I renamed them earlier as PAs after confirming/looking into their jsons (they say ““PhaseEncodingDirection”: “j-”,”).
Sorry for not catching the BIDS compliance-error earlier. I just now renamed the bval abd bvec files to comply with BIDS. and this is how the dwi directory looks like now:
Am i right in thinking that qsiprep will not run correctly because of this you said: “It looks like both your DWI and fmaps have the PA phase encoding direction, in which case they cannot be used to correct each other”?
I just tried again and it did give me the same error message as above.
Thank you again so much for your help!
Hi @gavkhar_a,
Sounds like fmap should be dir-AP then. Did you link the fieldmap to the DWI with the IntendedFor field? Did you confirm the DWI has a reverse phase encoding direction (i.e., j)?
The error doesn’t seem like it is about susceptibility distortion correction.
May you please show the BIDS validation report?
Best,
Steven
Thanks Steven! I could not find the IntendedFor characteristic in the json file – I may have missed it?
login-1:fmap$ cat sub-FACT130_ses-pre_dir-PA_epi.json
{
"AcquisitionMatrixPE": 128,
"AcquisitionNumber": 1,
"AcquisitionTime": "12:21:16.752500",
"BandwidthPerPixelPhaseEncode": 19.531,
"BaseResolution": 128,
"ConversionSoftware": "dcm2niix",
"ConversionSoftwareVersion": "v1.0.20180622 GCC6.3.0",
"DerivedVendorReportedEchoSpacing": 0.00080001,
"DeviceSerialNumber": "35469",
"DwellTime": 2.8e-06,
"EchoTime": 0.057,
"EffectiveEchoSpacing": 0.000400005,
"FlipAngle": 90,
"ImageOrientationPatientDICOM": [0.996543, 0.0807755, -0.0194357, -0.0824693, 0.990091, -0.113657],
"ImageType": [
"ORIGINAL",
"PRIMARY",
"M",
"ND",
"MOSAIC"
],
"InPlanePhaseEncodingDirectionDICOM": "COL",
"InstitutionAddress": "Vassar_43_Cambridge_MA_US_02139",
"InstitutionName": "McGovern_Brain_Institute",
"InstitutionalDepartmentName": "Department",
"MRAcquisitionType": "2D",
"MagneticFieldStrength": 3,
"Manufacturer": "Siemens",
"ManufacturersModelName": "TrioTim",
"Modality": "MR",
"ParallelReductionFactorInPlane": 2,
"PartialFourier": 0.75,
"PatientPosition": "HFS",
"PercentPhaseFOV": 100,
"PhaseEncodingDirection": "j-",
"PhaseEncodingSteps": 96,
"PhaseResolution": 1,
"PixelBandwidth": 1395,
"ProcedureStepDescription": "INVESTIGATORS_Gabrieli",
"ProtocolName": "TopUp_Distortion_Map_A-P-A_Diffusion",
"PulseSequenceDetails": "%CustomerSeq%_Andre_ep2d_se_dist_nav_map",
"ReceiveCoilName": "32Ch_Head",
"ReconMatrixPE": 128,
"RepetitionTime": 7.24,
"SAR": 0.212809,
"ScanOptions": "PFP_FS",
"ScanningSequence": "EP",
"SequenceName": "epse2d1_128",
"SequenceVariant": "SK_SP",
"SeriesDescription": "TopUp_Distortion_Map_A-P-A_Diffusion",
"SeriesNumber": 17,
"ShimSetting": [6351, -28080, -24530, 440, -193, 187, 88, 113],
"SliceThickness": 2,
"SliceTiming": [3.635, 0, 3.7325, 0.0975, 3.83, 0.1975, 3.93, 0.295, 4.0275, 0.3925, 4.125, 0.49, 4.2225, 0.59, 4.3225, 0.6875, 4.42, 0.785, 4.5175, 0.885, 4.6175, 0.9825, 4.715, 1.08, 4.8125, 1.1775, 4.91, 1.2775, 5.01, 1.375, 5.1075, 1.4725, 5.205, 1.5725, 5.305, 1.67, 5.4025, 1.7675, 5.5, 1.865, 5.5975, 1.965, 5.6975, 2.0625, 5.795, 2.16, 5.8925, 2.26, 5.9925, 2.3575, 6.09, 2.455, 6.1875, 2.555, 6.285, 2.6525, 6.385, 2.75, 6.4825, 2.8475, 6.58, 2.9475, 6.68, 3.045, 6.7775, 3.1425, 6.875, 3.2425, 6.975, 3.34, 7.0725, 3.4375, 7.17, 3.535],
"SoftwareVersions": "syngo_MR_B17",
"SpacingBetweenSlices": 2,
"StationName": "MRC35469",
"TaskName": "topupdwi",
"TotalReadoutTime": 0.0508007,
"TxRefAmp": 346.603}
And here is the bids-validator output. I may be wrong but this suggests it’s all compliant for this subject?
Thank you so much!
Gavkhar
Hi @gavkhar_a,
You have to manually add it to the fmap json. The entry should list the path to the DWI to correct relative to the subject folder (so the path beginning with ses-<>.
Please use the command line version tool instead.
Have you done this?
Best,
Steven
Hi Steven,
The dwi has “j-” direction so wouldn’t it be PA encoding? I was basing it off of this post of yours but please let me know if this varies depending on dwi and functional runs.
the bids validation output:
login-1:bids_validator$ python -m bids_validator /scratch/zt1/project/romeo-prj/shared/fact/sub-FACT130
login-1:bids_validator$ echo $?
0
login-1:bids_validator$
If you have any examples of how I would like the IntendedFor to fmap json, I'd appreciate it! thanks so much!
Best,
Gavkhar
Hi @gavkhar_a
If both your DWI and fmap are j- phase encoding direction they cannot be used for SDC.
“IntendedFor”: “ses-x/dwi/sub-y_ses-x_dwi.nii.gz”
I am not sure what python bids validator you are using. Please use the command line version: GitHub - bids-standard/bids-validator: Validator for the Brain Imaging Data Structure
Best,
Steven
Also can you clarify what those fmaps are? Were they created by TOPUP? That seems odd because that’s not something a scanner would produce (it requires off the scanner processing)
Thank you for the tip re bids-validator. I just ran it and here is the report (sorry it is long):
bids-validator-deno /scratch/zt1/project/romeo-prj/shared/fact/sub-FACT130
[WARNING] TSV_ADDITIONAL_COLUMNS_UNDEFINED A TSV file has extra columns which are not defined in its associated JSON sidecar
operator
/ses-pre/sub-FACT130_ses-pre_scans.tsv
/ses-post/sub-FACT130_ses-post_scans.tsv
randstr
/ses-pre/sub-FACT130_ses-pre_scans.tsv
/ses-post/sub-FACT130_ses-post_scans.tsv
TODO -- fill in rows and add more tab-separated columns if desired
/ses-post/dwi/sub-FACT130_ses-post_task-topupdwi_run-01_events.tsv
Please visit https://neurostars.org/search?q=TSV_ADDITIONAL_COLUMNS_UNDEFINED for existing conversations about this issue.
[WARNING] GZIP_HEADER_MTIME The gzip header contains a non-zero timestamp.
This may leak sensitive information or indicate a non-reproducible conversion process.
/ses-pre/dwi/sub-FACT130_ses-pre_dir-PA_epi.nii.gz
/ses-pre/anat/sub-FACT130_ses-pre_T1w.nii.gz
4 more files with the same issue
Please visit https://neurostars.org/search?q=GZIP_HEADER_MTIME for existing conversations about this issue.
[WARNING] SIDECAR_KEY_RECOMMENDED A data file's JSON sidecar is missing a key listed as recommended.
ReceiveCoilActiveElements
/ses-pre/dwi/sub-FACT130_ses-pre_dir-PA_epi.nii.gz - Field description: Information describing the active/selected elements of the receiver coil.
This does not correspond to a tag in the DICOM ontology.
The vendor-defined terminology for active coil elements can go in this field.
/ses-pre/anat/sub-FACT130_ses-pre_T1w.nii.gz - Field description: Information describing the active/selected elements of the receiver coil.
This does not correspond to a tag in the DICOM ontology.
The vendor-defined terminology for active coil elements can go in this field.
4 more files with the same issue
GradientSetType
/ses-pre/dwi/sub-FACT130_ses-pre_dir-PA_epi.nii.gz - Field description: It should be possible to infer the gradient coil from the scanner model.
If not, for example because of a custom upgrade or use of a gradient
insert set, then the specifications of the actual gradient coil should be
reported independently.
/ses-pre/anat/sub-FACT130_ses-pre_T1w.nii.gz - Field description: It should be possible to infer the gradient coil from the scanner model.
If not, for example because of a custom upgrade or use of a gradient
insert set, then the specifications of the actual gradient coil should be
reported independently.
4 more files with the same issue
MRTransmitCoilSequence
/ses-pre/dwi/sub-FACT130_ses-pre_dir-PA_epi.nii.gz - Field description: This is a relevant field if a non-standard transmit coil is used.
Corresponds to DICOM Tag 0018, 9049 `MR Transmit Coil Sequence`.
/ses-pre/anat/sub-FACT130_ses-pre_T1w.nii.gz - Field description: This is a relevant field if a non-standard transmit coil is used.
Corresponds to DICOM Tag 0018, 9049 `MR Transmit Coil Sequence`.
4 more files with the same issue
MatrixCoilMode
/ses-pre/dwi/sub-FACT130_ses-pre_dir-PA_epi.nii.gz - Field description: (If used)
A method for reducing the number of independent channels by combining in
analog the signals from multiple coil elements.
There are typically different default modes when using un-accelerated or
accelerated (for example, `"GRAPPA"`, `"SENSE"`) imaging.
/ses-pre/anat/sub-FACT130_ses-pre_T1w.nii.gz - Field description: (If used)
A method for reducing the number of independent channels by combining in
analog the signals from multiple coil elements.
There are typically different default modes when using un-accelerated or
accelerated (for example, `"GRAPPA"`, `"SENSE"`) imaging.
4 more files with the same issue
CoilCombinationMethod
/ses-pre/dwi/sub-FACT130_ses-pre_dir-PA_epi.nii.gz - Field description: Almost all fMRI studies using phased-array coils use root-sum-of-squares
(rSOS) combination, but other methods exist.
The image reconstruction is changed by the coil combination method
(as for the matrix coil mode above),
so anything non-standard should be reported.
/ses-pre/anat/sub-FACT130_ses-pre_T1w.nii.gz - Field description: Almost all fMRI studies using phased-array coils use root-sum-of-squares
(rSOS) combination, but other methods exist.
The image reconstruction is changed by the coil combination method
(as for the matrix coil mode above),
so anything non-standard should be reported.
4 more files with the same issue
PulseSequenceType
/ses-pre/dwi/sub-FACT130_ses-pre_dir-PA_epi.nii.gz - Field description: A general description of the pulse sequence used for the scan.
/ses-pre/anat/sub-FACT130_ses-pre_T1w.nii.gz - Field description: A general description of the pulse sequence used for the scan.
4 more files with the same issue
MTState
/ses-pre/dwi/sub-FACT130_ses-pre_dir-PA_epi.nii.gz - Field description: Boolean stating whether the magnetization transfer pulse is applied.
Corresponds to DICOM Tag 0018, 9020 `Magnetization Transfer`.
/ses-pre/anat/sub-FACT130_ses-pre_T1w.nii.gz - Field description: Boolean stating whether the magnetization transfer pulse is applied.
Corresponds to DICOM Tag 0018, 9020 `Magnetization Transfer`.
4 more files with the same issue
SpoilingState
/ses-pre/dwi/sub-FACT130_ses-pre_dir-PA_epi.nii.gz - Field description: Boolean stating whether the pulse sequence uses any type of spoiling
strategy to suppress residual transverse magnetization.
/ses-pre/anat/sub-FACT130_ses-pre_T1w.nii.gz - Field description: Boolean stating whether the pulse sequence uses any type of spoiling
strategy to suppress residual transverse magnetization.
4 more files with the same issue
NumberShots
/ses-pre/dwi/sub-FACT130_ses-pre_dir-PA_epi.nii.gz - Field description: The number of RF excitations needed to reconstruct a slice or volume
(may be referred to as partition).
Please mind that this is not the same as Echo Train Length which denotes the
number of k-space lines collected after excitation in a multi-echo readout.
The data type array is applicable for specifying this parameter before and
after the k-space center is sampled.
Please see
[`"NumberShots"` metadata field]\
(SPEC_ROOT/appendices/qmri.md#numbershots-metadata-field)
in the qMRI appendix for corresponding calculations.
/ses-pre/anat/sub-FACT130_ses-pre_T1w.nii.gz - Field description: The number of RF excitations needed to reconstruct a slice or volume
(may be referred to as partition).
Please mind that this is not the same as Echo Train Length which denotes the
number of k-space lines collected after excitation in a multi-echo readout.
The data type array is applicable for specifying this parameter before and
after the k-space center is sampled.
Please see
[`"NumberShots"` metadata field]\
(SPEC_ROOT/appendices/qmri.md#numbershots-metadata-field)
in the qMRI appendix for corresponding calculations.
4 more files with the same issue
ParallelReductionFactorOutOfPlane
/ses-pre/dwi/sub-FACT130_ses-pre_dir-PA_epi.nii.gz - Field description: The parallel imaging (for instance, GRAPPA) factor in the second phase encoding dimension of 3D sequences.
Use the denominator of the fraction of k-space encoded in the second phase encoding dimension.
For example, 2 means half of k-space is encoded.
Will typically be 1 for 2D sequences, as each slice in a 2D acquisition is usually fully encoded.
`ParallelReductionFactorOutOfPlane` should not be confused with `MultibandAccelerationFactor`,
as they imply different methods of accelerating the acquisition.
Corresponds to DICOM Tag 0018, 9155 `Parallel Reduction Factor out-of-plane`.
/ses-pre/anat/sub-FACT130_ses-pre_T1w.nii.gz - Field description: The parallel imaging (for instance, GRAPPA) factor in the second phase encoding dimension of 3D sequences.
Use the denominator of the fraction of k-space encoded in the second phase encoding dimension.
For example, 2 means half of k-space is encoded.
Will typically be 1 for 2D sequences, as each slice in a 2D acquisition is usually fully encoded.
`ParallelReductionFactorOutOfPlane` should not be confused with `MultibandAccelerationFactor`,
as they imply different methods of accelerating the acquisition.
Corresponds to DICOM Tag 0018, 9155 `Parallel Reduction Factor out-of-plane`.
4 more files with the same issue
ParallelAcquisitionTechnique
/ses-pre/dwi/sub-FACT130_ses-pre_dir-PA_epi.nii.gz - Field description: The type of parallel imaging used (for example `"GRAPPA"`, `"SENSE"`).
Corresponds to DICOM Tag 0018, 9078 `Parallel Acquisition Technique`.
/ses-pre/anat/sub-FACT130_ses-pre_T1w.nii.gz - Field description: The type of parallel imaging used (for example `"GRAPPA"`, `"SENSE"`).
Corresponds to DICOM Tag 0018, 9078 `Parallel Acquisition Technique`.
4 more files with the same issue
PartialFourierDirection
/ses-pre/dwi/sub-FACT130_ses-pre_dir-PA_epi.nii.gz - Field description: The direction where only partial Fourier information was collected.
Corresponds to DICOM Tag 0018, 9036 `Partial Fourier Direction`.
/ses-pre/anat/sub-FACT130_ses-pre_T1w.nii.gz - Field description: The direction where only partial Fourier information was collected.
Corresponds to DICOM Tag 0018, 9036 `Partial Fourier Direction`.
4 more files with the same issue
MixingTime
/ses-pre/dwi/sub-FACT130_ses-pre_dir-PA_epi.nii.gz - Field description: In the context of a stimulated- and spin-echo 3D EPI sequence for B1+ mapping
or a stimulated-echo MRS sequence,
corresponds to the interval between spin- and stimulated-echo pulses.
In the context of a diffusion-weighted double spin-echo sequence,
corresponds to the interval between two successive diffusion sensitizing
gradients, specified in seconds.
/ses-pre/anat/sub-FACT130_ses-pre_T1w.nii.gz - Field description: In the context of a stimulated- and spin-echo 3D EPI sequence for B1+ mapping
or a stimulated-echo MRS sequence,
corresponds to the interval between spin- and stimulated-echo pulses.
In the context of a diffusion-weighted double spin-echo sequence,
corresponds to the interval between two successive diffusion sensitizing
gradients, specified in seconds.
4 more files with the same issue
InversionTime
/ses-pre/dwi/sub-FACT130_ses-pre_dir-PA_epi.nii.gz - Field description: The inversion time (TI) for the acquisition, specified in seconds.
Inversion time is the time after the middle of inverting RF pulse to middle
of excitation pulse to detect the amount of longitudinal magnetization.
Corresponds to DICOM Tag 0018, 0082 `Inversion Time`
(please note that the DICOM term is in milliseconds not seconds).
/ses-pre/anat/sub-FACT130_ses-pre_T1w.nii.gz - Field description: The inversion time (TI) for the acquisition, specified in seconds.
Inversion time is the time after the middle of inverting RF pulse to middle
of excitation pulse to detect the amount of longitudinal magnetization.
Corresponds to DICOM Tag 0018, 0082 `Inversion Time`
(please note that the DICOM term is in milliseconds not seconds).
4 more files with the same issue
SliceEncodingDirection
/ses-pre/dwi/sub-FACT130_ses-pre_dir-PA_epi.nii.gz - Field description: The axis of the NIfTI data along which slices were acquired,
and the direction in which `"SliceTiming"` is defined with respect to.
`i`, `j`, `k` identifiers correspond to the first, second and third axis of
the data in the NIfTI file.
A `-` sign indicates that the contents of `"SliceTiming"` are defined in
reverse order - that is, the first entry corresponds to the slice with the
largest index, and the final entry corresponds to slice index zero.
When present, the axis defined by `"SliceEncodingDirection"` needs to be
consistent with the `slice_dim` field in the NIfTI header.
When absent, the entries in `"SliceTiming"` must be in the order of increasing
slice index as defined by the NIfTI header.
/ses-pre/fmap/sub-FACT130_ses-pre_dir-PA_epi.nii.gz - Field description: The axis of the NIfTI data along which slices were acquired,
and the direction in which `"SliceTiming"` is defined with respect to.
`i`, `j`, `k` identifiers correspond to the first, second and third axis of
the data in the NIfTI file.
A `-` sign indicates that the contents of `"SliceTiming"` are defined in
reverse order - that is, the first entry corresponds to the slice with the
largest index, and the final entry corresponds to slice index zero.
When present, the axis defined by `"SliceEncodingDirection"` needs to be
consistent with the `slice_dim` field in the NIfTI header.
When absent, the entries in `"SliceTiming"` must be in the order of increasing
slice index as defined by the NIfTI header.
2 more files with the same issue
MultibandAccelerationFactor
/ses-pre/dwi/sub-FACT130_ses-pre_dir-PA_epi.nii.gz - Field description: The multiband factor, for multiband acquisitions.
/ses-pre/anat/sub-FACT130_ses-pre_T1w.nii.gz - Field description: The multiband factor, for multiband acquisitions.
4 more files with the same issue
EffectiveEchoSpacing
/ses-pre/anat/sub-FACT130_ses-pre_T1w.nii.gz - Field description: The "effective" sampling interval, specified in seconds,
between lines in the phase-encoding direction,
defined based on the size of the reconstructed image in the phase direction.
It is frequently, but incorrectly, referred to as "dwell time"
(see the `"DwellTime"` parameter for actual dwell time).
It is required for unwarping distortions using field maps.
Note that beyond just in-plane acceleration,
a variety of other manipulations to the phase encoding need to be accounted
for properly, including partial fourier, phase oversampling,
phase resolution, phase field-of-view and interpolation.
/ses-post/anat/sub-FACT130_ses-post_T1w.nii.gz - Field description: The "effective" sampling interval, specified in seconds,
between lines in the phase-encoding direction,
defined based on the size of the reconstructed image in the phase direction.
It is frequently, but incorrectly, referred to as "dwell time"
(see the `"DwellTime"` parameter for actual dwell time).
It is required for unwarping distortions using field maps.
Note that beyond just in-plane acceleration,
a variety of other manipulations to the phase encoding need to be accounted
for properly, including partial fourier, phase oversampling,
phase resolution, phase field-of-view and interpolation.
PhaseEncodingDirection
/ses-pre/anat/sub-FACT130_ses-pre_T1w.nii.gz - Field description: The letters `i`, `j`, `k` correspond to the first, second and third axis of
the data in the NIFTI file.
The polarity of the phase encoding is assumed to go from zero index to
maximum index unless `-` sign is present
(then the order is reversed - starting from the highest index instead of
zero).
`PhaseEncodingDirection` is defined as the direction along which phase is was
modulated which may result in visible distortions.
Note that this is not the same as the DICOM term
`InPlanePhaseEncodingDirection` which can have `ROW` or `COL` values.
/ses-post/anat/sub-FACT130_ses-post_T1w.nii.gz - Field description: The letters `i`, `j`, `k` correspond to the first, second and third axis of
the data in the NIFTI file.
The polarity of the phase encoding is assumed to go from zero index to
maximum index unless `-` sign is present
(then the order is reversed - starting from the highest index instead of
zero).
`PhaseEncodingDirection` is defined as the direction along which phase is was
modulated which may result in visible distortions.
Note that this is not the same as the DICOM term
`InPlanePhaseEncodingDirection` which can have `ROW` or `COL` values.
TotalReadoutTime
/ses-pre/anat/sub-FACT130_ses-pre_T1w.nii.gz - Field description: This is actually the "effective" total readout time,
defined as the readout duration, specified in seconds,
that would have generated data with the given level of distortion.
It is NOT the actual, physical duration of the readout train.
If `"EffectiveEchoSpacing"` has been properly computed,
it is just `EffectiveEchoSpacing * (ReconMatrixPE - 1)`.
/ses-post/anat/sub-FACT130_ses-post_T1w.nii.gz - Field description: This is actually the "effective" total readout time,
defined as the readout duration, specified in seconds,
that would have generated data with the given level of distortion.
It is NOT the actual, physical duration of the readout train.
If `"EffectiveEchoSpacing"` has been properly computed,
it is just `EffectiveEchoSpacing * (ReconMatrixPE - 1)`.
B0FieldIdentifier
/ses-pre/fmap/sub-FACT130_ses-pre_dir-PA_epi.nii.gz - Field description: The presence of this key states that this particular 3D or 4D image MAY be
used for fieldmap estimation purposes.
Each `"B0FieldIdentifier"` MUST be a unique string within one participant's tree,
shared only by the images meant to be used as inputs for the estimation of a
particular instance of the *B<sub>0</sub> field* estimation.
It is RECOMMENDED to derive this identifier from DICOM Tags, for example,
DICOM tag 0018, 1030 `Protocol Name`, or DICOM tag 0018, 0024 `Sequence Name`
when the former is not defined (for example, in GE devices.)
/ses-pre/fmap/sub-FACT130_ses-pre_dir-PA_epi.nii.gz - Field description: The presence of this key states that this particular 3D or 4D image MAY be
used for fieldmap estimation purposes.
Each `"B0FieldIdentifier"` MUST be a unique string within one participant's tree,
shared only by the images meant to be used as inputs for the estimation of a
particular instance of the *B<sub>0</sub> field* estimation.
It is RECOMMENDED to derive this identifier from DICOM Tags, for example,
DICOM tag 0018, 1030 `Protocol Name`, or DICOM tag 0018, 0024 `Sequence Name`
when the former is not defined (for example, in GE devices.)
StimulusPresentation
/ses-post/dwi/sub-FACT130_ses-post_task-topupdwi_run-01_events.tsv - Field description: Object containing key-value pairs related to the software used to present
the stimuli during the experiment.
Please visit https://neurostars.org/search?q=SIDECAR_KEY_RECOMMENDED for existing conversations about this issue.
[WARNING] EVENTS_TSV_MISSING Task scans should have a corresponding 'events.tsv' file.
If this is a resting state scan you can ignore this warning or rename the task to include the word "rest".
/ses-post/dwi/sub-FACT130_ses-post_task-topupdwi_run-01_bold.nii.gz
Please visit https://neurostars.org/search?q=EVENTS_TSV_MISSING for existing conversations about this issue.
[ERROR] MISSING_DATASET_DESCRIPTION A dataset_description.json file is required in the root of the dataset
Please visit https://neurostars.org/search?q=MISSING_DATASET_DESCRIPTION for existing conversations about this issue.
[ERROR] INVALID_LOCATION The file has a valid name, but is located in an invalid directory.
/ses-pre/sub-FACT130_ses-pre_scans.tsv - Expected location: /sub-FACT130/ses-pre/
/ses-pre/dwi/sub-FACT130_ses-pre_dir-PA_epi.bvec - Expected location: /sub-FACT130/ses-pre/
16 more files with the same issue
Please visit https://neurostars.org/search?q=INVALID_LOCATION for existing conversations about this issue.
[ERROR] SCANS_FILENAME_NOT_MATCH_DATASET Filenames in scans.tsv file do not match what is present in the BIDS dataset.
/ses-pre/sub-FACT130_ses-pre_scans.tsv
/ses-post/sub-FACT130_ses-post_scans.tsv
Please visit https://neurostars.org/search?q=SCANS_FILENAME_NOT_MATCH_DATASET for existing conversations about this issue.
[ERROR] DATATYPE_MISMATCH The datatype directory does not match datatype of found suffix and extension
/ses-pre/dwi/sub-FACT130_ses-pre_dir-PA_epi.bvec - Datatype rule being applied: rules.files.raw.fmap.pepolar
/ses-pre/dwi/sub-FACT130_ses-pre_dir-PA_epi.bval - Datatype rule being applied: rules.files.raw.fmap.pepolar
4 more files with the same issue
Please visit https://neurostars.org/search?q=DATATYPE_MISMATCH for existing conversations about this issue.
[ERROR] VOLUME_COUNT_MISMATCH The number of volumes in this scan does not match the number of volumes in the
associated '.bvec' and '.bval' files.
/ses-pre/dwi/sub-FACT130_ses-pre_dir-PA_epi.nii.gz
Please visit https://neurostars.org/search?q=VOLUME_COUNT_MISMATCH for existing conversations about this issue.
[ERROR] ALL_FILENAME_RULES_HAVE_ISSUES Multiple filename rules were found as potential matches. All of them had at least one issue during filename validation.
/ses-post/dwi/sub-FACT130_ses-post_task-topupdwi_run-01_events.tsv - Rules that matched with issues: rules.files.raw.task.events, rules.files.raw.task.events__mri, rules.files.raw.task.events__motion, rules.files.raw.task.events__pet, rules.files.raw.task.events__mrs
Please visit https://neurostars.org/search?q=ALL_FILENAME_RULES_HAVE_ISSUES for existing conversations about this issue.
Summary: Available Tasks: Available Modalities:
19 Files, 58.8 MB topupdwi MRI
1 - Subjects 2 - Sessions
If you have any questions, please post on https://neurostars.org/tags/bids.
login-1:~$
Hi @gavkhar_a,
Yes these errors are precluding qsiprep.
.epi is not a bids valid suffix for DWI
You should see if this is consistent for all subjects, and how large the discrepancy is. I recommend using most up to date dcm2niix when converting subjects. The version you are using is 5 years old.
DWI files do not have event files or task labels.
Best,
Steven
Ah got it! Thank you so so much Steven!
Best,
Gavkhar
Hi Steven, thanks again. The QSIprep ran for about an hour and exited with some errors. I attached a slurm file as well as crash text files it outputted.
Do you have an insight into what might be the issue?
Also, it seems to be failing to grab bval and bvec files but I made sure they are there so wondering why. Thanks in advance!
(also, Rachel says hi 
)
slurm.txt (108.7 KB)
crash-20250820-142819-ga2541-get_template_image-27617b43-8cd8-4648-a21b-e4d05802cfcd.txt (5.5 KB)
crash-20250820-141417-ga2541-merge_dwis-b6e27e93-92f3-4f8e-8b5c-d8852171aac5.txt (4.7 KB)
Hi @gavkhar_a,
What is the current BIDS validator output and tree directory structure for this subject?
For this error:
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node get_template_image.
Traceback:
Traceback (most recent call last):
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/urllib3/connection.py", line 196, in _new_conn
sock = connection.create_connection(
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/urllib3/util/connection.py", line 85, in create_connection
raise err
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/urllib3/util/connection.py", line 73, in create_connection
sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/urllib3/connectionpool.py", line 789, in urlopen
response = self._make_request(
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/urllib3/connectionpool.py", line 490, in _make_request
raise new_e
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/urllib3/connectionpool.py", line 466, in _make_request
self._validate_conn(conn)
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1095, in _validate_conn
conn.connect()
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/urllib3/connection.py", line 615, in connect
self.sock = sock = self._new_conn()
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/urllib3/connection.py", line 205, in _new_conn
raise ConnectTimeoutError(
urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPSConnection object at 0x1537cd0d0a60>, 'Connection to templateflow.s3.amazonaws.com timed out. (connect timeout=None)')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/requests/adapters.py", line 667, in send
resp = conn.urlopen(
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/urllib3/connectionpool.py", line 843, in urlopen
retries = retries.increment(
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/urllib3/util/retry.py", line 519, in increment
raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='templateflow.s3.amazonaws.com', port=443): Max retries exceeded with url: /tpl-MNI152NLin2009cAsym/tpl-MNI152NLin2009cAsym_res-01_T1w.nii.gz (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x1537cd0d0a60>, 'Connection to templateflow.s3.amazonaws.com timed out. (connect timeout=None)'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 401, in run
runtime = self._run_interface(runtime)
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsiprep/interfaces/anatomical.py", line 226, in _run_interface
template_path = get_template(
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/templateflow/conf/__init__.py", line 69, in wrapper
return func(*args, **kwargs)
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/templateflow/api.py", line 145, in get
_s3_get(filepath)
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/templateflow/api.py", line 299, in _s3_get
r = requests.get(url, stream=True)
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/requests/api.py", line 73, in get
return request("get", url, params=params, **kwargs)
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/requests/adapters.py", line 688, in send
raise ConnectTimeout(e, request=request)
requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='templateflow.s3.amazonaws.com', port=443): Max retries exceeded with url: /tpl-MNI152NLin2009cAsym/tpl-MNI152NLin2009cAsym_res-01_T1w.nii.gz (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x1537cd0d0a60>, 'Connection to templateflow.s3.amazonaws.com timed out. (connect timeout=None)'))
It looks like your compute nodes do not have internet connection, so your jobs fail when you try to download the anatomical templates from TemplateFlow. To get around this:
conda install -c conda-forge datalad)datalad clone https://github.com/templateflow/templateflow.git)cd to your templateflow directory, and pull the necessary files from the cloud# if you see no folders in the templateflow directory first run the below
datalad get -n *
# if you see folders then run the below
datalad get tpl-MNI152NLin2009cAsym
-B /path/to/your/templateflow:/templateflow to your apptainer run preamble. Also add export APPTAINERENV_TEMPLATEFLOW_HOME=/templateflow before your apptainer run command.For the other error:
Traceback (most recent call last):
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py", line 66, in run_node
result["result"] = node.run(updatehash=updatehash)
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 525, in run
result = self._run_interface(execute=True)
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 643, in _run_interface
return self._run_command(execute)
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 769, in _run_command
raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node merge_dwis.
Traceback:
Traceback (most recent call last):
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 401, in run
runtime = self._run_interface(runtime)
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsiprep/interfaces/dwi_merge.py", line 76, in _run_interface
to_concat, b0_means, corrections = harmonize_b0s(
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsiprep/interfaces/dwi_merge.py", line 700, in harmonize_b0s
b0_mean = index_img(dwi_nii, b0_indices).get_fdata().mean()
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nilearn/image/image.py", line 669, in index_img
return _index_img(imgs, index)
File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nilearn/_utils/niimg_conversions.py", line 77, in _index_img
img, _get_data(img)[:, :, :, index], img.affine, copy_header=True
IndexError: index 4 is out of bounds for axis 3 with size 4
Can you print the bvals for this subject? I see the following line in the crash log:
scan_metadata = {'/data/sub-FACT130/ses-pre/dwi/sub-FACT130_ses-pre_dwi.nii.gz': {'AcquisitionMatrixPE': 128, 'AcquisitionNumber': 1, 'AcquisitionTime': '12:21:16.752500', 'BandwidthPerPixelPhaseEncode': 19.531, 'BaseResolution': 128, 'ConversionSoftware': 'dcm2niix', 'ConversionSoftwareVersion': 'v1.0.20180622 GCC6.3.0', 'DerivedVendorReportedEchoSpacing': 0.00080001, 'DeviceSerialNumber': '35469', 'DwellTime': 2.8e-06, 'EchoTime': 0.057, 'EffectiveEchoSpacing': 0.000400005, 'FlipAngle': 90, 'ImageOrientationPatientDICOM': [0.996543, 0.0807755, -0.0194357, -0.0824693, 0.990091, -0.113657], 'ImageType': ['ORIGINAL', 'PRIMARY', 'M', 'ND', 'MOSAIC'], 'InPlanePhaseEncodingDirectionDICOM': 'COL', 'InstitutionAddress': 'Vassar_43_Cambridge_MA_US_02139', 'InstitutionName': 'McGovern_Brain_Institute', 'InstitutionalDepartmentName': 'Department', 'MRAcquisitionType': '2D', 'MagneticFieldStrength': 3, 'Manufacturer': 'Siemens', 'ManufacturersModelName': 'TrioTim', 'Modality': 'MR', 'ParallelReductionFactorInPlane': 2, 'PartialFourier': 0.75, 'PatientPosition': 'HFS', 'PercentPhaseFOV': 100, 'PhaseEncodingDirection': 'j-', 'PhaseEncodingSteps': 96, 'PhaseResolution': 1, 'PixelBandwidth': 1395, 'ProcedureStepDescription': 'INVESTIGATORS_Gabrieli', 'ProtocolName': 'TopUp_Distortion_Map_A-P-A_Diffusion', 'PulseSequenceDetails': '%CustomerSeq%_Andre_ep2d_se_dist_nav_map', 'ReceiveCoilName': '32Ch_Head', 'ReconMatrixPE': 128, 'RepetitionTime': 7.24, 'SAR': 0.212809, 'ScanOptions': 'PFP_FS', 'ScanningSequence': 'EP', 'SequenceName': 'epse2d1_128', 'SequenceVariant': 'SK_SP', 'SeriesDescription': 'TopUp_Distortion_Map_A-P-A_Diffusion', 'SeriesNumber': 17, 'ShimSetting': [6351, -28080, -24530, 440, -193, 187, 88, 113], 'SliceThickness': 2, 'SliceTiming': [3.635, 0, 3.7325, 0.0975, 3.83, 0.1975, 3.93, 0.295, 4.0275, 0.3925, 4.125, 0.49, 4.2225, 0.59, 4.3225, 0.6875, 4.42, 0.785, 4.5175, 0.885, 4.6175, 0.9825, 4.715, 1.08, 4.8125, 1.1775, 4.91, 1.2775, 5.01, 1.375, 5.1075, 1.4725, 5.205, 1.5725, 5.305, 1.67, 5.4025, 1.7675, 5.5, 1.865, 5.5975, 1.965, 5.6975, 2.0625, 5.795, 2.16, 5.8925, 2.26, 5.9925, 2.3575, 6.09, 2.455, 6.1875, 2.555, 6.285, 2.6525, 6.385, 2.75, 6.4825, 2.8475, 6.58, 2.9475, 6.68, 3.045, 6.7775, 3.1425, 6.875, 3.2425, 6.975, 3.34, 7.0725, 3.4375, 7.17, 3.535], 'SoftwareVersions': 'syngo_MR_B17', 'SpacingBetweenSlices': 2, 'StationName': 'MRC35469', 'TaskName': 'topupdwi', 'TotalReadoutTime': 0.0508007, 'TxRefAmp': 346.603}}
Based on that scan metadata, it looks like that is some topupdwi file? Is it possible you are inputting the wrong file into qsiprep? Can you explain where those topup images came from? Where they something right off the scanner? Are they just one volume?
Hi Rachel!
Best,
Steven
Thank you!
j-- encoding. This means PA encoding, correct? But could be AP as you previously said? I think I confused myself with this at this point…
Thanks!
Gavkhar
sub-FACT130_ses-pre_dir-PA_epi.txt (2.9 KB)
Hi @gavkhar_a,
For the future, text is preferable to screenshots.
If you could provide the rest of the details requested in my last post (BIDS validator, bvals, information about the topupdwi file), that would help me debug.
You can use miniforge which installs only a minimal installation of mamba.
Yes, but you would still need datalad at some point to retrieve the images from the cloud.
Best,
Steven
Hi @Steven I ran the bids-validator on another subject just to check things around and these are the errors it is throwing me that I can’t understand where they are coming from as it seems looking fine folder-tree wise:
[WARNING] GZIP_HEADER_MTIME The gzip header contains a non-zero timestamp.
This may leak sensitive information or indicate a non-reproducible conversion process.
/ses-pre/fmap/sub-FACT181_ses-pre_dir-AP_epi.nii.gz
/ses-pre/func/sub-FACT181_ses-pre_task-story_run-01_bold.nii.gz
4 more files with the same issue
Please visit https://neurostars.org/search?q=GZIP_HEADER_MTIME for existing conversations about this issue.
[WARNING] EVENTS_TSV_MISSING Task scans should have a corresponding 'events.tsv' file.
If this is a resting state scan you can ignore this warning or rename the task to include the word "rest".
/ses-pre/func/sub-FACT181_ses-pre_task-story_run-01_bold.nii.gz
Please visit https://neurostars.org/search?q=EVENTS_TSV_MISSING for existing conversations about this issue.
[ERROR] MISSING_DATASET_DESCRIPTION A dataset_description.json file is required in the root of the dataset
Please visit https://neurostars.org/search?q=MISSING_DATASET_DESCRIPTION for existing conversations about this issue.
[ERROR] ALL_FILENAME_RULES_HAVE_ISSUES Multiple filename rules were found as potential matches. All of them had at least one issue during filename validation.
/ses-pre/sub-FACT181_ses-pre_task-story_run-01_events.tsv - Rules that matched with issues: rules.files.raw.task.events, rules.files.raw.task.events__mri, rules.files.raw.task.events__motion, rules.files.raw.task.events__pet, rules.files.raw.task.events__mrs
Please visit https://neurostars.org/search?q=ALL_FILENAME_RULES_HAVE_ISSUES for existing conversations about this issue.
[ERROR] INVALID_LOCATION The file has a valid name, but is located in an invalid directory.
/ses-pre/sub-FACT181_ses-pre_scans.tsv - Expected location: /sub-FACT181/ses-pre/
/ses-pre/fmap/sub-FACT181_ses-pre_dir-AP_epi.nii.gz - Expected location: /sub-FACT181/ses-pre/
15 more files with the same issue
Please visit https://neurostars.org/search?q=INVALID_LOCATION for existing conversations about this issue.
[ERROR] SCANS_FILENAME_NOT_MATCH_DATASET Filenames in scans.tsv file do not match what is present in the BIDS dataset.
/ses-pre/sub-FACT181_ses-pre_scans.tsv
Please visit https://neurostars.org/search?q=SCANS_FILENAME_NOT_MATCH_DATASET for existing conversations about this issue.
Summary: Available Tasks: Available Modalities:
18 Files, 101 MB topupdwi MRI
1 - Subjects 1 - Sessions story
rest
topuprest
Here is the bids-tree for this subject:
login-3:sub-FACT181$ ls
ses-pre
login-3:sub-FACT181$ cd ses-pre
login-3:ses-pre$ ls
anat dwi fmap func sub-FACT181_ses-pre_scans.tsv sub-FACT181_ses-pre_task-story_run-01_events.tsv
login-3:ses-pre$ cd func
login-3:func$ ls
sub-FACT181_ses-pre_task-rest_run-01_bold.json sub-FACT181_ses-pre_task-story_run-01_bold.nii.gz
sub-FACT181_ses-pre_task-rest_run-01_bold.nii.gz sub-FACT181_ses-pre_task-topuprest_run-01_bold.json
sub-FACT181_ses-pre_task-rest_run-01_events.tsv sub-FACT181_ses-pre_task-topuprest_run-01_bold.nii.gz
sub-FACT181_ses-pre_task-story_run-01_bold.json sub-FACT181_ses-pre_task-topuprest_run-01_events.tsv
login-3:func$ cd ..
login-3:ses-pre$ cd dwi
login-3:dwi$ ls
sub-FACT181_ses-pre_dwi.bval sub-FACT181_ses-pre_dwi.bvec sub-FACT181_ses-pre_dwi.json sub-FACT181_ses-pre_dwi.nii.gz
login-3:dwi$
I am struggling to understand these errors as I checked and all seems to be located in places it is asking for. All other errors I was able to resolve but this is where I am stuck at.
Thanks much !!
Gavkhar
Hi @gavkhar_a,
It is hard to diagnose without knowing the contents of your scans.tsv file. But that file is optional and you might just be better off deleting it (or at least putting it somewhere else temporarily).
The events file does not belong in the session folder. Events files are only valid for certain data types (e.g… BOLD).
I see a topuprest file in the func folder. Are you sure that’s not supposed to be an fmap?
You need a dataset_description.json file in the root of your dataset.
Best,
Steven
Thanks @Steven ! I put the scan.tsv file in scan_tsv_files for now.
The dataset-description.json file exists in the main folder where other subjects folder are (here is the excerpt-screenshot):
I reran the validator and it is not complaining about the topuprest file anymore but I guess in any case, this should now potentially impede QSIPrep…?
The only error that is left and does not make sense to me is this:
[ERROR] INVALID_LOCATION The file has a valid name, but is located in an invalid directory.
/ses-pre/sub-FACT181_ses-pre_dir-AP_epi.nii.gz - Expected location: /sub-FACT181/ses-pre/
/ses-pre/sub-FACT181_ses-pre_dir-AP_epi.json - Expected location: /sub-FACT181/ses-pre/
because the those files are already there:
login-3:fmap$ ls
login-3:fmap$ cd ..
login-3:ses-pre$ ls
anat dwi fmap func sub-FACT181_ses-pre_dir-AP_epi.json sub-FACT181_ses-pre_dir-AP_epi.nii.gz
login-3:ses-pre$
And the fmap folder is not empty so I am not sure how it will find fmaps for dwi now
Hi @gavkhar_a,
the scans tsv files folder should not be placed within the bids root because it is not a valid folder name. You can add the folder to a .bidsignore file temporarily.
They should not be there, as they are field maps, and should be in fmap.
Best,
Steven
Hi @Steven, I’ve been finally been able to run QSIprep on one subject and it’s been running for almost 25 hours by now which made me a bit concerned so I looked at the slurm file. From what I could gather, it looks like the last update is from yesterday at around noon:14 min and there is no print-out saying that QSIPrep is completed, etc.
Does this mean it is somehow stuck in a loop? Or could there be another issue at hand?
Slurm file attached. Thank you so so much for all your help – truly couldn’t have gone this far without your support, Steven!
Best,
Gavkhar
slurm-13637849.txt (71.3 KB)
MANAGED BY INCF