ValueError: Fieldmap-less (SyN) estimation was requested, but PhaseEncodingDirection information appears to be absent. fMRIprep v. 22.0.2

Hi,

I tried adding the -B option. The fMRIprep did run successfully but there were no results. If you could please explain it a little bit more of -B function, it would be great.

Thank you!

When singularity begins a container, no data from your machine goes inside of it by default (except for your home drive). So if you want fMRIPrep to be able to read from / write to a drive, you need to mount it with the -B argument. All relevant drives should be mounted (some people may have their bids root and scratch directory on different file system). You can mount multiple drives with -B /drive1,/drive2 etc.

Best,
Steven

Hi,

I added the -B arguments. The script ran successfully but it still showed “susceptibility distortion correction: none” in the report.

Here is the script

#User inputs:
bids_root_dir=/scratch/g/jgoveas/R01_nii/baseline_mid/PROJECT/bidstest5/
subj=3172101920
nthreads=4
mem=40 #gb
container=singularity #docker or singularity

#Begin:

#Convert virtual memory from gb to mb
mem=echo "${mem//[!0-9]/}"
mem_mb=echo $(((mem*1000)-5000))

export FS_LICENSE=/scratch/g/jgoveas/R01_nii/baseline_mid/bids/license.txt \
singularity run
-B /scratch/g/jgoveas/R01_nii/baseline_mid
-B /scratch/g/jgoveas/R01_nii/baseline_mid/PROJECT
-B /scratch/g/jgoveas/R01_nii/baseline_mid/PROJECT/bidstest5
–cleanenv /scratch/g/jgoveas/R01_nii/baseline_mid/fmriprep-22.0.2.simg $bids_root_dir $bids_root_dir/derivatives
participant
–participant-label $subj
–skip-bids-validation
–md-only-boilerplate
–fs-license-file /scratch/g/jgoveas/R01_nii/baseline_mid/bids/license.txt
–fs-no-reconall
–output-spaces MNI152NLin2009cAsym:res-2
–output-spaces func:res-native
–output-spaces MNI152NLin2009cAsym:res-native
–nthreads $nthreads
–mem_mb mem_mb \ --ignore slicetiming \ -w /scratch/g/jgoveas/R01_nii/baseline_mid/TOPUP_{subj}

Here is the tree hierarchy:
sub-3172101920
└── ses-1
├── anat
│ ├── sub-3172101920_ses-1_FLAIR.json
│ ├── sub-3172101920_ses-1_FLAIR.nii
│ ├── sub-3172101920_ses-1_T1w.json
│ └── sub-3172101920_ses-1_T1w.nii
├── fmap
│ ├── sub-3172101920_ses-1_acq-mid1_dir-AP_epi.json
│ ├── sub-3172101920_ses-1_acq-mid1_dir-AP_epi.nii
│ ├── sub-3172101920_ses-1_acq-mid1_dir-PA_epi.json
│ ├── sub-3172101920_ses-1_acq-mid1_dir-PA_epi.nii
│ ├── sub-3172101920_ses-1_acq-mid2_dir-AP_epi.json
│ ├── sub-3172101920_ses-1_acq-mid2_dir-AP_epi.nii
│ ├── sub-3172101920_ses-1_acq-mid2_dir-PA_epi.json
│ └── sub-3172101920_ses-1_acq-mid2_dir-PA_epi.nii
└── func
├── sub-3172101920_ses-1_task-mid1_bold.json
├── sub-3172101920_ses-1_task-mid1_bold.nii
├── sub-3172101920_ses-1_task-mid2_bold.json
└── sub-3172101920_ses-1_task-mid2_bold.nii

Here is one of the fmap.json
{
“Modality”: “MR”,
“MagneticFieldStrength”: 3,
“ImagingFrequency”: 127.772,
“Manufacturer”: “GE”,
“PulseSequenceName”: “epi”,
“InternalPulseSequenceName”: “EPI”,
“ManufacturersModelName”: “SIGNA Premier”,
“InstitutionName”: “MCW Premier”,
“DeviceSerialNumber”: “00000000FLORIO1X”,
“StationName”: “MCWPREM”,
“BodyPartExamined”: “BRAIN”,
“PatientPosition”: “HFS”,
“ProcedureStepDescription”: “GovGriefO1”,
“SoftwareVersions”: “27\LX\MR Software release:RX27.0_R03_1850.a”,
“MRAcquisitionType”: “2D”,
“SeriesDescription”: “SE Map PE2”,
“ProtocolName”: “Goveas_GovGriefR01_10052”,
“ScanningSequence”: “EP\RM”,
“SequenceVariant”: “NONE”,
“ScanOptions”: “SAT_GEMS\MP_GEMS\EPI_GEMS\FS”,
“ImageType”: [“ORIGINAL”, “PRIMARY”, “OTHER”],
“SeriesNumber”: 26,
“AcquisitionTime”: “10:20:1.000000”,
“AcquisitionNumber”: 1,
“TriggerDelayTime”: 14677,
“SliceThickness”: 2.4,
“SpacingBetweenSlices”: 2.4,
“SAR”: 0.20231,
“EchoTime”: 0.08,
“RepetitionTime”: 7.4,
“FlipAngle”: 90,
“PhaseEncodingPolarityGE”: “Unflipped”,
“CoilString”: “RM:32NovaHead2”,
“PercentPhaseFOV”: 100,
“PercentSampling”: 100,
“AcquisitionMatrixPE”: 90,
“ReconMatrixPE”: 128,
“EffectiveEchoSpacing”: 0.000403654,
“TotalReadoutTime”: 0.051264,
“IntendedFor”: “ses-1/func/sub-3172101920_ses-1_task-mid1_bold.nii”,
“PixelBandwidth”: 3906.25,
“PhaseEncodingDirection”: “j-”,
“SliceTiming”: [
0,
3.7,
0.123333,
3.82333,
0.246667,
3.94667,
0.37,
4.07,
0.493333,
4.19333,
0.616667,
4.31667,
0.74,
4.44,
0.863333,
4.56333,
0.986667,
4.68667,
1.11,
4.81,
1.23333,
4.93333,
1.35667,
5.05667,
1.48,
5.18,
1.60333,
5.30333,
1.72667,
5.42667,
1.85,
5.55,
1.97333,
5.67333,
2.09667,
5.79667,
2.22,
5.92,
2.34333,
6.04333,
2.46667,
6.16667,
2.59,
6.29,
2.71333,
6.41333,
2.83667,
6.53667,
2.96,
6.66,
3.08333,
6.78333,
3.20667,
6.90667,
3.33,
7.03,
3.45333,
7.15333,
3.57667,
7.27667 ],
“ImageOrientationPatientDICOM”: [
1,
-0,
0,
-0,
1,
0 ],
“InPlanePhaseEncodingDirectionDICOM”: “COL”,
“ConversionSoftware”: “dcm2niix”,
“ConversionSoftwareVersion”: “v1.0.20210317”
}

Please let me know what is missing.

Thank you for your help.

You need the phase encoding direction set in your BOLD json file to use SYN-SDC.

Hi,

Thank you for your help. I have “PhaseEncodingDirection”: “j-”, in the BOLD.json. Is this ok or do I need to modify it?

Thanks again.

If you want to use your fmap/_epi images for Fieldmap correction (since in your last version of your script you don’t use --ignore fieldmaps), you need to provide the "IntendedFor" tag in the fmap/epi.json files or "B0FieldSource"/"B0FieldIdentifier" tags in func.json and fmap/epi.json files respectively.

Hi,

I added "B0FieldSource" /"B0FieldIdentifier" tags in func.json and fmap/epi.json files respectively and still have "IntendedFor" tag in the fmap/epi.json files. However, the susceptibility distortion correction is still not run.

Note: Bids validated. I cleaned the work folder every time. The data was collected using GE scanner.

Here is the fmap.json:

"EffectiveEchoSpacing": 0.000403654,
"TotalReadoutTime": 0.051264,
    "IntendedFor": "ses-1/func/sub-3172101920_ses-1_task-mid1_bold.nii",
"PixelBandwidth": 3906.25,
"PhaseEncodingDirection": "j-",
    "B0FieldIdentifier": "pepolar_fmap0",

Here is the corresponding Bold.json:

"EffectiveEchoSpacing": 0.000588944,
"TotalReadoutTime": 0.052416,
"PixelBandwidth": 5555.56,
"PhaseEncodingDirection": "j-",
    "B0FieldSource": "pepolar_fmap0",

Here is the fmriprep command:

singularity run \
-B /scratch/g/jgoveas/R01_nii/baseline_mid \
-B /scratch/g/jgoveas/R01_nii/baseline_mid/PROJECT \
-B /scratch/g/jgoveas/R01_nii/baseline_mid/PROJECT/bidstest5 \
--cleanenv /scratch/g/jgoveas/R01_nii/baseline_mid/fmriprep-22.0.2.simg \
$bids_root_dir $bids_root_dir/derivatives \
participant \
--participant-label $subj \
--skip-bids-validation \
--md-only-boilerplate \
--fs-license-file /scratch/g/jgoveas/R01_nii/baseline_mid/bids/license.txt \
--fs-no-reconall \
--output-spaces MNI152NLin2009cAsym:res-2 \
--nthreads $nthreads \
--mem_mb $mem_mb \
--ignore slicetiming \
-w /scratch/g/jgoveas/R01_nii/baseline_mid/TOPUP_${subj}

Thank you for your help.

Thank you for providing those details. We must be still missing something here.

  • I guess you correctly wrote the B0FiledIdentifier and ÌntendedFor fields in both AP and PA _epi.json files in the fmap/ folder?

  • Did you check that dir-AP_epi.json and dir-PA_epi.json files have actually opposite phase encoding direction (i.e "-j"and "j"respectively)?

  • Why do you keep --skip-bids-validationoption in your command? Would you get an error if you remove it?

  • Would you mind sharing the beginning of the output log of FMRIPREP, where details are thrown about the detection of fieldmaps and the identification of functional runs to be corrected for SDC?

Thank you for being patient with me.

Yes, I believed that I have the B0FiledIdentifier and ÌntendedFor fields in both AP and PA _epi.json files in the fmap/ folder; and the AP and PA were also correct.

I reran with --skip-bids-validation. There were some warning regarding missing README file. The fmriprep ran successfully otherwise.

Below is the beginning of the output log. There were nothing about sdc or fieldmaps. (nothing showed up when using the find function for those two keywords either).

221030-09:34:17,968 nipype.workflow IMPORTANT:
Building fMRIPrep’s workflow:
* BIDS dataset path: /scratch/g/jgoveas/R01_nii/baseline_mid/PROJECT/bidstest5.
* Participant list: [‘3172101920’].
* Run identifier: 20221030-093359_ba671b65-2560-4ecb-afdc-1dbd47b3ca7a.
* Output spaces: MNI152NLin2009cAsym:res-2.
* Pre-run FreeSurfer’s SUBJECTS_DIR: /scratch/g/jgoveas/R01_nii/baseline_mid/PROJECT/bidstest5/derivatives/sourcedata/freesurfer.
221030-09:34:19,758 nipype.workflow INFO:
No single-band-reference found for sub-3172101920_ses-1_task-mid1_bold.nii.
221030-09:34:19,977 nipype.workflow INFO:
No single-band-reference found for sub-3172101920_ses-1_task-mid2_bold.nii.
221030-09:34:23,182 nipype.workflow INFO:
fMRIPrep workflow graph with 522 nodes built successfully.
221030-09:34:36,142 nipype.workflow IMPORTANT:
fMRIPrep started!
221030-09:34:39,390 nipype.workflow INFO:
[Node] Setting-up “fmriprep_22_0_wf.single_subject_3172101920_wf.anat_preproc_wf.brain_extraction_wf.full_wm” in “/scratch/g/jgoveas/R01_nii/baseline_mid/TOPUP2_3172101920/fmriprep_22_0_wf/single_subject_3172101920_wf/anat_preproc_wf/brain_extraction_wf/full_wm”.
221030-09:34:39,413 nipype.workflow INFO:
[Node] Executing “full_wm” <nipype.interfaces.utility.wrappers.Function>
221030-09:34:39,570 nipype.workflow INFO:
[Node] Setting-up “fmriprep_22_0_wf.single_subject_3172101920_wf.anat_preproc_wf.brain_extraction_wf.lap_tmpl” in “/scratch/g/jgoveas/R01_nii/baseline_mid/TOPUP2_3172101920/fmriprep_22_0_wf/single_subject_3172101920_wf/anat_preproc_wf/brain_extraction_wf/lap_tmpl”.
221030-09:34:39,573 nipype.workflow INFO:
[Node] Setting-up “fmriprep_22_0_wf.single_subject_3172101920_wf.anat_preproc_wf.brain_extraction_wf.res_tmpl” in “/scratch/g/jgoveas/R01_nii/baseline_mid/TOPUP2_3172101920/fmriprep_22_0_wf/single_subject_3172101920_wf/anat_preproc_wf/brain_extraction_wf/res_tmpl”.
221030-09:34:39,592 nipype.workflow INFO:
[Node] Executing “res_tmpl” <niworkflows.interfaces.nibabel.RegridToZooms>
221030-09:34:39,592 nipype.workflow INFO:
[Node] Executing “lap_tmpl” <nipype.interfaces.ants.utils.ImageMath>
221030-09:34:40,113 nipype.workflow INFO:
[Node] Finished “full_wm”, elapsed time 0.690009s.

Thank you.

Being only a regular user of FMRIPREP, I am running out of suggestions to offer. One last thing, which should be unrelated, is the presence of --fs-no-reconall which was not advised to use earlier by @Steven.

When you run the bids validator, does fmap shows up as one available modality?

It looks like the fmap folder is not even seen by FMRIPREP!

I would be curious to try a minimal sample of your dataset on my side!

Yep, the bids validator does see fmap folder. Yes, I totally agree. It seems that fMRIprep doesn’t see it at all.

When I used the previous version (20.2.3) to run with fmap or without (–use-syn-sdc), both worked.

I still wonder what “ValueError: Fieldmap-less (SyN) estimation was requested, but PhaseEncodingDirection information appears to be absent.” mean when using --use-syn-sdc.

That would be great if you can try running it. Thanks!

Are you absolutely sure you are using fMRIPrep 22.0.2? Because as you can see in the changelog (What’s new — fmriprep version documentation) there was a bug with using SYN and --ignore-fieldmaps that was not patched until 22.0.2. If you run fmriprep -v using your 22.0.2 container, does the terminal return 22.0.2?

Hi,

Yep, it shows 22.0.2.

Thank you.

Hi @Nuttaon , I played with your dataset and indeed, you may have uncovered a bug!

I run your dataset with either FMRIPREP v21.02 or v22.02, and in one case (v21.0.2) SDC correction is run, and in the other case (v22.0.2) , it is not run!

I used exactly the same dataset, and the same script, just changing the singularity image and the name of the working directory, the output directory and the freesurfer directory.

singularity run -B /scratch/jsein/BIDS:/work,$HOME/.templateflow:/opt/templateflow --cleanenv /scratch/jsein/my_images/fmriprep-22.0.2.simg \
		 --fs-license-file /work/freesurfer/license.txt /work/$study /work/$study/derivatives/fmriprep22  \
		 participant --participant-label $sub \
		 -w /work/temp_data22_${study}\
		 --mem-mb 50000 --omp-nthreads 10 --nthreads 12  \
		 --fd-spike-threshold 0.5 --dvars-spike-threshold 2.0 --bold2t1w-dof 6 --debug fieldmaps --write-graph  \
		 --output-spaces T1w MNI152NLin2009cAsym --ignore slicetiming  --fs-subjects-dir /work/$study/derivatives/fmriprep22/sourcedata/freesurfer \

or

singularity run -B /scratch/jsein/BIDS:/work,$HOME/.templateflow:/opt/templateflow --cleanenv /scratch/jsein/my_images/fmriprep-21.0.2.simg \
		 --fs-license-file /work/freesurfer/license.txt /work/$study /work/$study/derivatives/fmriprep21  \
		 participant --participant-label $sub \
		 -w /work/temp_data21_${study}\
		 --mem-mb 50000 --omp-nthreads 10 --nthreads 12  \
		 --fd-spike-threshold 0.5 --dvars-spike-threshold 2.0 --bold2t1w-dof 6 --debug fieldmaps --write-graph  \
		 --output-spaces T1w MNI152NLin2009cAsym --ignore slicetiming  --fs-subjects-dir /work/$study/derivatives/fmriprep21/sourcedata/freesurfer \

It is very strange, I don’t understand what is going on.

I can’t wrap my mind around the problem right now:

  • What is sure is that FMRIPREP v21.0.2 systematically “detects” the fieldmaps.
  • I had previously used FMRIPREP 22.0.2 where the fieldmaps were correctly detected. But If I create a new directory in which I copy-paste a subject from the project that worked, and I re-run FMRIPREP v22.0.2 (same code that worked previously) on this new project (but the same subject!), the field maps are not detected anymore!
    -Crazier: if now I copy the subject from @Nuttaon into the BIDS directory for which FMRIPREP v22.0.2 was detecting the fieldmaps, it does detect the fieldmap!!!

I don’t know FMRIPREP deep enough to know what is the heuristic for the selection of the workflows, to debug further.

Hi @jsein ,

Thank you so much for testing this! I still have no luck. My experience is that FMRIPREP version 20.2.3 consistently detects the fieldmaps. While, version 22.0.2 never detected them (I tried singularity, docker, and python on two different platforms: OS and Linux, and on different task and different subjects). I also tried using the updated dcm2niix.

This is a silly question. Can it be dataset_description.json?

I went down the rabbit hole: what makes FMRIPREP v22.0.2 not “recognize” the fieldmaps is the fact that in your dataset_description.json , you use:

    "DatasetType": "derivative",
    "GeneratedBy": [
        {
            "Name": "Manual"
        }
    ],

I don’t know what is the meaning of these two fields, and how they are related to the behavior of the fieldmap detection by FMRIPREP v22.0.2, but it is fact. If you remove those fields from dataset_description.json, the SDC correction is run by FMRIPREP v 22.0.2!

This does not affect FMRIPREP v21.0.2 or earlier.

A long topic but I learnt something in the process.

This DatasetType defined in dataset_description.json defines the “scope” and how pipeline with interact with the dataset.

If we look here:

line: 271:

    base_entities = {
        "subject": subject,
        "extension": [".nii", ".nii.gz"],
        "scope": "raw",  # Ensure derivatives are not captured
    }

This may suggest that SDCFlows does not deal with datasets whose “scope” (The "DataType" defined in dataset_description.json) is "derivative"`.

PyBIS seems to be involved in this too. Well, a whole new world…

Hi @jsein,

Thanks you so much for looking into this and helping me solve this issue! I learnt a lot during this whole process.

Nuttaon

Hey, just wanted to say that this is still an issue for me in fMRI prep v 23.0.2. I never would have even considered this issue if I hadn’t read it here. Is it fixable?

Jen