RuntimeError: EchoTime2 metadata field not found

scho0011 · March 26, 2019, 12:47am

Hi all,

I am running fmriprep on some longitudinal resting state data and am getting the following error for all subjects/sessions:

in_file = /home/fmriprep/data/fmriprep_work/sub-008/fmriprep_wf/single_subject_008_wf/func_preproc_ses_1_task_rest_wf/sdc_wf/phdiff_wf/cleanup_wf/AddEdge/sub-008_ses-1_phasediff_rads_unwrapped_filt_demean_maths.nii.gz
metadata = {}

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/multiproc.py", line 69, in run_node
    result['result'] = node.run(updatehash=updatehash)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 473, in run
    result = self._run_interface(execute=True)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 557, in _run_interface
    return self._run_command(execute)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 637, in _run_command
    result = self._interface.run(cwd=outdir)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 369, in run
    runtime = self._run_interface(runtime)
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/interfaces/fmap.py", line 205, in _run_interface
    _delta_te(self.inputs.metadata),
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/interfaces/fmap.py", line 530, in _delta_te
    'EchoTime2 metadata field not found. Please consult the BIDS specification.')
RuntimeError: EchoTime2 metadata field not found. Please consult the BIDS specification.

However, my data is in BIDS format (Validated) and with IntendedFor, EchoTime1 and EchoTime2 fields specifed in the phaseDiff JSON file. Please see below for a file tree of a typical subject and a typical PhaseDiff JSON file:

sub-008/
├── ses-1
│   ├── anat
│   │   ├── sub-008_ses-1_T1w.json
│   │   ├── sub-008_ses-1_T1w.nii
│   │   ├── sub-008_ses-1_T2w.json
│   │   └── sub-008_ses-1_T2w.nii
│   ├── fmap
│   │   ├── sub-008_ses-1_magnitude1.json
│   │   ├── sub-008_ses-1_magnitude1.nii
│   │   ├── sub-008_ses-1_magnitude2.json
│   │   ├── sub-008_ses-1_magnitude2.nii
│   │   ├── sub-008_ses-1_phasediff.json
│   │   └── sub-008_ses-1_phasediff.nii
│   └── func
│       ├── sub-008_ses-1_task-rest_bold.json
│       └── sub-008_ses-1_task-rest_bold.nii
├── ses-2
│   ├── anat
│   │   ├── sub-008_ses-2_T1w.json
│   │   ├── sub-008_ses-2_T1w.nii
│   │   ├── sub-008_ses-2_T2w.json
│   │   └── sub-008_ses-2_T2w.nii
│   ├── fmap
│   │   ├── sub-008_ses-2_magnitude1.json
│   │   ├── sub-008_ses-2_magnitude1.nii
│   │   ├── sub-008_ses-2_magnitude2.json
│   │   ├── sub-008_ses-2_magnitude2.nii
│   │   ├── sub-008_ses-2_phasediff.json
│   │   └── sub-008_ses-2_phasediff.nii
│   └── func
│       ├── sub-008_ses-2_task-rest_bold.json
│       └── sub-008_ses-2_task-rest_bold.nii
└── ses-3
    ├── anat
    │   ├── sub-008_ses-3_T1w.json
    │   ├── sub-008_ses-3_T1w.nii
    │   ├── sub-008_ses-3_T2w.json
    │   └── sub-008_ses-3_T2w.nii
    ├── fmap
    │   ├── sub-008_ses-3_magnitude1.json
    │   ├── sub-008_ses-3_magnitude1.nii
    │   ├── sub-008_ses-3_magnitude2.json
    │   ├── sub-008_ses-3_magnitude2.nii
    │   ├── sub-008_ses-3_phasediff.json
    │   └── sub-008_ses-3_phasediff.nii
    └── func
        ├── sub-008_ses-3_task-rest_bold.json
        └── sub-008_ses-3_task-rest_bold.nii

Typical JSON file corresponding to phasediff image:

{
	"Manufacturer": "Siemens",
	"ManufacturersModelName": "TrioTim",
	"ProcedureStepDescription": "MRI_RESEARCH_STUDY",
	"ProtocolName": "gre_field_mapping",
	"ImageType": ["ORIGINAL", "PRIMARY", "P", "ND"],
	"AcquisitionDateTime": "2012-08-13T10:39:47.015625",
	"MagneticFieldStrength": 3,
	"FlipAngle": 60,
	"EchoTime1": 0.00519,
	"EchoTime2": 0.00765,
	"RepetitionTime": 0.42,
	"PhaseEncodingDirection": "j-",
	"ConversionSoftware": "dcm2niix",
	"ConversionSoftwareVersion": "v1.0.20170207",
	"IntendedFor": "ses-1/func/sub-008_ses-1_task-rest_bold.nii"
}

Thanks for your help on this, and please let me know if more information is required.

franklin · March 26, 2019, 1:08pm

Hi @scho0011

May you please share your fMRIPrep command and does the validator also pass when you initially start to run fMRIPrep?

scho0011 · March 26, 2019, 10:50pm

Hi @franklin

Thanks for your quick reply. The fmriprep (version: 1.3.0_post2) command I am using is:

fmriprep --participant_label ${subject} --output-space {T1w,template,fsnative,fsaverage5} --use-aroma --longitudinal --mem_mb 80000 -n-cpus 8 --skull-strip-template OASIS30ANTs --fs-license-file ${fslicense}  -w ${work_dir} ${bids_dir} ${output_dir} participant

While the initial validator when I run fMRIprep states “This dataset appears to be BIDS compatible.” It is preceded by the following warning (repeated 47 times):

"dim[0] is out-of-range, we'll simply try continuing to read the file, but this will most likely fail horribly."

Thanks, Franklin.

oesteban · March 27, 2019, 3:23pm

Hi @scho0011, could you also share the output of fMRIPrep? I’m afraid this is a bug that should be fixed. I believe that the warning you are seeing comes from some AFNI tool unrelated to the field map processing, so you can dismiss it.

scho0011 · March 28, 2019, 12:05am

Hi @oesteban,

Please find the output and log file of a typical subject here: https://github.com/sidchop/FMRIprep/

Is there a way to dismiss the error so FMRIprep will continue rather than stop?

Regards,
Sid

oesteban · March 28, 2019, 7:10pm

Thanks very much. Now I have a clearer picture of your problem.

fMRIPrep crashes because the node extracting the dTE parameter is getting an empty dictionary for metadata. Can you dig up the following text file and post the contents?:
```
/home/fmriprep/data/fmriprep_work/sub-008/fmriprep_wf/single_subject_008_wf/func_preproc_ses_1_task_rest_wf/sdc_wf/phdiff_wf/meta/_report/report.rst
```
The "dim[0] is out-of-range, we'll simply try continuing to read the file, but this will most likely fail horribly." message seems to come from the BIDS-Validator, so I’m going to page @rwblair here for his thoughts. However, it does not seem to break processing and you are probably safe ignoring it.

You can use --ignore fieldmaps to skip susceptibility distortion correction. However, it is very unlikely you want to do that. Alternatively, you can try removing the IntendedFor field from the json files and use --use-syn --force-syn to apply fieldmap-less correction. I’d recommend first assessing the problem because it probably has an easy, less involved solution than modifying the original dataset.

scho0011 · March 31, 2019, 4:13am

Hi @oesteban,

I have added the report.rst file for you: https://github.com/sidchop/FMRIprep

Thanks,
Sid

gspitz · March 31, 2019, 12:53pm

Dear all,

I am also receiving the same error–see attached error fig.
Best,
Gershoon.

rwblair · April 1, 2019, 8:14pm

Unfortunately I’m not seeing the error in the bids-validator source code:

(rblair@PSY-C02Q70WEFY10) :) (13:09)
~/projects/bids-validator $ grep -nRF "fail horribly" .
(rblair@PSY-C02Q70WEFY10) :( 1 (13:09)
~/projects/bids-validator $

Also not seeing it in fmriprep, smriprep, or pybids. I wonder where this error’s parents are.

oesteban · April 1, 2019, 9:56pm

I’d bet that comes from some AFNI tool and that standard error pipe gets flushed before others.

@gspitz, could you please provide the contents of

/vols/Scratch/gspitz/gut_study/data/fmriprep_wd_6/fmriprep_wf/single_subject_018_wf/func_preproc_ses_002_task_rest_echo_1_wf/sdc_wf/phdiff_wf/meta/_report/report.rst

@scho0011, I can see that the node that should extract metadata from your input is actually failing. I’ll try to understand why this is happening.

oesteban · April 1, 2019, 10:24pm

@scho0011, @gspitz are you not using containers?

Could you post the outputs of

fmriprep --version

python -c 'import niworkflows; print(niworkflows.__version__)'

and

python -c 'import bids; print(bids.__version__)'

?

scho0011 · April 1, 2019, 11:42pm

@oesteban, I am useing fmriprep on a computing cluster, where it is loaded as a module.
Here are the requested outputs:

module load fmriprep
fmriprep --version
fmriprep v1.3.0.post2

module load python 
python -c 'import niworkflows; print(niworkflows.__version__)'
0.0.6-3

python -c 'import bids; print(bids.__version__)'
0.1

Thanks,
Sid

oesteban · April 1, 2019, 11:44pm

Hi Sid,

With those versions I’m surprised it worked at all! Please request your admins to install appropriate versions of fmriprep dependencies (and, since they are at that, it’d be nice for them tu upgrade fmriprep itself too).

Cheers,
Oscar

scho0011 · April 1, 2019, 11:53pm

Hi Oscar,

Okay thanks for letting me know - will request an upgrade. Also, the cluster has more versions of python, ranging from python/2.7.11 to python/3.7.2 - I guess it might be worth trying loading these before I run the fmriprep command.

Cheers,
Sid

oesteban · April 2, 2019, 1:33am

fMRIPrep won’t work for python < 3.5. However, if the fmriprep module is well designed, it should load an appropriate version of python.

gspitz · April 2, 2019, 8:52am

Dear @oesteban,

Thanks for following up. Attached is the sdc meta report.

I have run fmriprep --version on my docker container

I am using version v1.3.2

I assume because I am using a docker container with singularity the other two version of niworkflows and bids do not apply as all of the dependencies should be in the .sif image?

For example, the version of niworkflows within my fmriprep singularity container is: 0.8.1
The bids version is: 0.7.1

Kind regards,
Gershon.

I am using singularity to run a docker container. My command I use to run fmriprep is also attached.
fmriprep_launch.txt (436 Bytes)
sdc_meta_report.txt (3.1 KB)

gspitz · April 5, 2019, 8:34am

Hi all (@oesteban),

I am still encountering this same issue after having run the analysis in a number of different ways. Is there something about the magnitude or phase fmap images that might be causing this? I have attached the json files as well as the phdiff meta file. sub-018_ses-001_phasediff.txt (23.0 KB) phdiff_w_meta_report.txt (3.1 KB)

This analysis used singularity and FMRIPrep image:

FMRIPrep version: 1.3.2
FMRIPrep command: /usr/local/miniconda/bin/fmriprep /home/fs0/gspitz/scratch/gut_study/data/bids_nii/ /home/fs0/gspitz/scratch/gut_study/data/fmriprep_out_15 participant -w /home/fs0/gspitz/scratch/gut_study/data/fmriprep_wd_15 --participant-label 018 --t2s-coreg --mem_mb 80000 --n_cpus 8 --use-aroma --verbose --fs-license-file /home/fs0/gspitz/scratch/gut_study/data/license.txt

Kind regards,
Gershon.

oesteban · April 6, 2019, 12:28am

Can you post how your BIDS dataset tree looks like? (i.e. the tree command).

gspitz · April 6, 2019, 12:58pm

Thanks @oesteban,

I also realised that BIDS as well as fmriprep accept phase reversed epi for SDC. I am trying this now, instead of phasediff image to see if this works.

Tree attached:
bids_tree.txt (7.3 KB)

gspitz · April 7, 2019, 11:39am

Quick update – running epi-based SDC worked! So it seems as though the error is specific to the phase diff method.
Best,
Gershon