ValueError: embedded null byte; during fmriprep 1.0.0-rc10

I’m not sure what to make of an error I’m encountering while re-running the latest version of fmriprep (1.0.0-rc10). This is also my first time converting the dicoms with heudiconv (previously, I had just been calling dcm2niix directly), so that is another confound (the dataset did at least pass the online bids validator).

Call:

module load singularity/singularity-2.4

singularity run poldracklab_fmriprep_latest-2017-11-10-3e683742db3e.img /data/vtf/ /data/vtf/derivatives/ participant -w /data/vtf/derivatives/work/ --fs-license-file /license.txt --output-space template T1w fsaverage fsnative --nthreads 14 --mem_mb 143360 --participant-label 01 02

(note: singularity image created with singularityware/docker2singularity)

output (the derivatives/fmriprep/logs folder is empty, and the derivatives/fmriprep/sub-01 folder hadn’t been created):

Running fMRIPREP version 1.0.0-rc10:

  • Participant list: [‘01’, ‘02’].
  • Run identifier: 20171115-191753_190b20ab-5d74-49a1-af7f-e84deeb76876.

171115-19:18:40,665 workflow WARNING:
No fieldmaps found or they were ignored, building base workflow for dataset /home/ps52a/data/vtf/sub-01/func/sub-01_task-con_run-01_bold.nii.gz.
171115-19:18:50,894 workflow WARNING:
No fieldmaps found or they were ignored, building base workflow for dataset /home/ps52a/data/vtf/sub-01/func/sub-01_task-con_run-02_bold.nii.gz.
171115-19:19:01,160 workflow WARNING:
No fieldmaps found or they were ignored, building base workflow for dataset /home/ps52a/data/vtf/sub-01/func/sub-01_task-con_run-03_bold.nii.gz.
171115-19:19:11,423 workflow WARNING:
No fieldmaps found or they were ignored, building base workflow for dataset /home/ps52a/data/vtf/sub-01/func/sub-01_task-con_run-04_bold.nii.gz.
171115-19:19:21,667 workflow WARNING:
No fieldmaps found or they were ignored, building base workflow for dataset /home/ps52a/data/vtf/sub-01/func/sub-01_task-con_run-05_bold.nii.gz.
171115-19:19:31,769 workflow WARNING:
No fieldmaps found or they were ignored, building base workflow for dataset /home/ps52a/data/vtf/sub-01/func/sub-01_task-con_run-06_bold.nii.gz.
171115-19:19:42,29 workflow WARNING:
No fieldmaps found or they were ignored, building base workflow for dataset /home/ps52a/data/vtf/sub-01/func/sub-01_task-con_run-07_bold.nii.gz.
171115-19:19:52,341 workflow WARNING:
No fieldmaps found or they were ignored, building base workflow for dataset /home/ps52a/data/vtf/sub-01/func/sub-01_task-con_run-08_bold.nii.gz.
171115-19:20:02,202 workflow WARNING:
No fieldmaps found or they were ignored, building base workflow for dataset /home/ps52a/data/vtf/sub-01/func/sub-01_task-loc_run-01_bold.nii.gz.
171115-19:20:09,539 workflow WARNING:
No fieldmaps found or they were ignored, building base workflow for dataset /home/ps52a/data/vtf/sub-01/func/sub-01_task-loc_run-02_bold.nii.gz.
171115-19:20:19,487 workflow WARNING:
No fieldmaps found or they were ignored, building base workflow for dataset /home/ps52a/data/vtf/sub-01/func/sub-01_task-ret_run-01_bold.nii.gz.
171115-19:20:29,653 workflow WARNING:
No fieldmaps found or they were ignored, building base workflow for dataset /home/ps52a/data/vtf/sub-01/func/sub-01_task-ret_run-02_bold.nii.gz.
171115-19:20:45,5 workflow WARNING:
No fieldmaps found or they were ignored, building base workflow for dataset /home/ps52a/data/vtf/sub-02/func/sub-02_task-con_run-01_bold.nii.gz.
171115-19:20:51,784 workflow WARNING:
No fieldmaps found or they were ignored, building base workflow for dataset /home/ps52a/data/vtf/sub-02/func/sub-02_task-con_run-02_bold.nii.gz.
171115-19:21:01,9 workflow WARNING:
No fieldmaps found or they were ignored, building base workflow for dataset /home/ps52a/data/vtf/sub-02/func/sub-02_task-con_run-03_bold.nii.gz.
171115-19:21:09,802 workflow WARNING:
No fieldmaps found or they were ignored, building base workflow for dataset /home/ps52a/data/vtf/sub-02/func/sub-02_task-con_run-04_bold.nii.gz.
171115-19:21:19,411 workflow WARNING:
No fieldmaps found or they were ignored, building base workflow for dataset /home/ps52a/data/vtf/sub-02/func/sub-02_task-con_run-05_bold.nii.gz.
171115-19:21:27,558 workflow WARNING:
No fieldmaps found or they were ignored, building base workflow for dataset /home/ps52a/data/vtf/sub-02/func/sub-02_task-con_run-06_bold.nii.gz.
171115-19:21:33,881 workflow WARNING:
No fieldmaps found or they were ignored, building base workflow for dataset /home/ps52a/data/vtf/sub-02/func/sub-02_task-con_run-07_bold.nii.gz.
171115-19:21:43,527 workflow WARNING:
No fieldmaps found or they were ignored, building base workflow for dataset /home/ps52a/data/vtf/sub-02/func/sub-02_task-con_run-08_bold.nii.gz.
171115-19:21:52,763 workflow WARNING:
No fieldmaps found or they were ignored, building base workflow for dataset /home/ps52a/data/vtf/sub-02/func/sub-02_task-loc_run-01_bold.nii.gz.
171115-19:22:02,10 workflow WARNING:
No fieldmaps found or they were ignored, building base workflow for dataset /home/ps52a/data/vtf/sub-02/func/sub-02_task-loc_run-02_bold.nii.gz.
171115-19:22:11,251 workflow WARNING:
No fieldmaps found or they were ignored, building base workflow for dataset /home/ps52a/data/vtf/sub-02/func/sub-02_task-ret_run-01_bold.nii.gz.
Traceback (most recent call last):
File “/usr/local/miniconda/bin/fmriprep”, line 11, in
load_entry_point(‘fmriprep==1.0.0rc10’, ‘console_scripts’, ‘fmriprep’)()
File “/usr/local/miniconda/lib/python3.6/site-packages/fmriprep/cli/run.py”, line 228, in main
create_workflow(opts)
File “/usr/local/miniconda/lib/python3.6/site-packages/fmriprep/cli/run.py”, line 347, in create_workflow
fmriprep_wf.run(**plugin_settings)
File “/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/workflows.py”, line 591, in run
runner.run(execgraph, updatehash=updatehash, config=self.config)
File “/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/plugins/base.py”, line 182, in run
self._send_procs_to_workers(updatehash=updatehash, graph=graph)
File “/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/plugins/multiproc.py”, line 260, in _send_procs_to_workers
if self._local_hash_check(jobid, graph):
File “/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/plugins/base.py”, line 333, in _local_hash_check
hash_exists, _, _, _ = self.procs[jobid].hash_exists()
File “/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/nodes.py”, line 278, in hash_exists
hashed_inputs, hashvalue = self._get_hashval()
File “/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/nodes.py”, line 445, in _get_hashval
hash_method=self.config[‘execution’][‘hash_method’])
File “/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/interfaces/base.py”, line 572, in get_hashval
hash_files=hash_files)))
File “/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/interfaces/base.py”, line 587, in _get_sorteddict
hash_files=hash_files)))
File “/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/interfaces/base.py”, line 587, in _get_sorteddict
hash_files=hash_files)))
File “/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/interfaces/base.py”, line 587, in _get_sorteddict
hash_files=hash_files)))
File “/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/interfaces/base.py”, line 594, in _get_sorteddict
hash_files=hash_files))
File “/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/interfaces/base.py”, line 600, in _get_sorteddict
os.path.isfile(objekt)):
File “/usr/local/miniconda/lib/python3.6/genericpath.py”, line 30, in isfile
st = os.stat(path)
ValueError: embedded null byte

This looks from a first glance like it may be a corrupted file in your working directory. Clearing it out and starting over may fix the issue. However, there’s not enough information to be sure, and even so, how this happened is unclear. Am I right in thinking that this was not the first run in the working directory? And is there any chance that you had two jobs running in parallel (possibly on different nodes) using the same working directory?

And just to cover a couple bases: Was this working on an earlier version of fmriprep? And have you looked at your files to make sure there aren’t any unusual names?

Thanks for the response. Error persists after clearing out the working directory (but you are correct that this wasn’t the first time running in that folder) and rerunning with freshly converted files. It wasn’t the case that two jobs were running in parallel.

And just to cover a couple bases: Was this working on an earlier version of fmriprep? And have you looked at your files to make sure there aren’t any unusual names?

When converted with a call to dcm2niix directly, an earlier version of fmriprep (1.0.0-rc8) had finished. 1.0.0-rc10 also finishes with those same files. No obviously looking weird file names (output of tree below).

Are there any fields in the json files that I should be confirming were set correctly? Or, is it more likely that one of the nifti files was corrupted.


tree 

vtf/
├── CHANGES
├── code
│ ├── convert.txt
│ ├── heuristic_ntf.py
│ ├── heuristic_ntf.pyc
│ └── run_all.sh
├── dataset_description.json
├── participants.tsv
├── README
├── sub-01
│ ├── anat
│ │ ├── sub-01_T1w.json
│ │ └── sub-01_T1w.nii.gz
│ ├── func
│ │ ├── sub-01_task-con_run-01_bold.json
│ │ ├── sub-01_task-con_run-01_bold.nii.gz
│ │ ├── sub-01_task-con_run-02_bold.json
│ │ ├── sub-01_task-con_run-02_bold.nii.gz
│ │ ├── sub-01_task-con_run-03_bold.json
│ │ ├── sub-01_task-con_run-03_bold.nii.gz
│ │ ├── sub-01_task-con_run-04_bold.json
│ │ ├── sub-01_task-con_run-04_bold.nii.gz
│ │ ├── sub-01_task-con_run-05_bold.json
│ │ ├── sub-01_task-con_run-05_bold.nii.gz
│ │ ├── sub-01_task-con_run-06_bold.json
│ │ ├── sub-01_task-con_run-06_bold.nii.gz
│ │ ├── sub-01_task-con_run-07_bold.json
│ │ ├── sub-01_task-con_run-07_bold.nii.gz
│ │ ├── sub-01_task-con_run-08_bold.json
│ │ ├── sub-01_task-con_run-08_bold.nii.gz
│ │ ├── sub-01_task-loc_run-01_bold.json
│ │ ├── sub-01_task-loc_run-01_bold.nii.gz
│ │ ├── sub-01_task-loc_run-02_bold.json
│ │ ├── sub-01_task-loc_run-02_bold.nii.gz
│ │ ├── sub-01_task-ret_run-01_bold.json
│ │ ├── sub-01_task-ret_run-01_bold.nii.gz
│ │ ├── sub-01_task-ret_run-02_bold.json
│ │ └── sub-01_task-ret_run-02_bold.nii.gz
│ ├── sub-01_scans.tsv
│ ├── sub-01_task-loc_events.tsv
│ └── sub-01_task-ret_events.tsv
├── sub-02
│ ├── anat
│ │ ├── sub-02_T1w.json
│ │ └── sub-02_T1w.nii.gz
│ ├── func
│ │ ├── sub-02_task-con_run-01_bold.json
│ │ ├── sub-02_task-con_run-01_bold.nii.gz
│ │ ├── sub-02_task-con_run-01_events.tsv
│ │ ├── sub-02_task-con_run-02_bold.json
│ │ ├── sub-02_task-con_run-02_bold.nii.gz
│ │ ├── sub-02_task-con_run-02_events.tsv
│ │ ├── sub-02_task-con_run-03_bold.json
│ │ ├── sub-02_task-con_run-03_bold.nii.gz
│ │ ├── sub-02_task-con_run-03_events.tsv
│ │ ├── sub-02_task-con_run-04_bold.json
│ │ ├── sub-02_task-con_run-04_bold.nii.gz
│ │ ├── sub-02_task-con_run-04_events.tsv
│ │ ├── sub-02_task-con_run-05_bold.json
│ │ ├── sub-02_task-con_run-05_bold.nii.gz
│ │ ├── sub-02_task-con_run-05_events.tsv
│ │ ├── sub-02_task-con_run-06_bold.json
│ │ ├── sub-02_task-con_run-06_bold.nii.gz
│ │ ├── sub-02_task-con_run-06_events.tsv
│ │ ├── sub-02_task-con_run-07_bold.json
│ │ ├── sub-02_task-con_run-07_bold.nii.gz
│ │ ├── sub-02_task-con_run-07_events.tsv
│ │ ├── sub-02_task-con_run-08_bold.json
│ │ ├── sub-02_task-con_run-08_bold.nii.gz
│ │ ├── sub-02_task-con_run-08_events.tsv
│ │ ├── sub-02_task-loc_run-01_bold.json
│ │ ├── sub-02_task-loc_run-01_bold.nii.gz
│ │ ├── sub-02_task-loc_run-01_events.tsv
│ │ ├── sub-02_task-loc_run-02_bold.json
│ │ ├── sub-02_task-loc_run-02_bold.nii.gz
│ │ ├── sub-02_task-loc_run-02_events.tsv
│ │ ├── sub-02_task-ret_run-01_bold.json
│ │ ├── sub-02_task-ret_run-01_bold.nii.gz
│ │ └── sub-02_task-ret_run-01_events.tsv
│ └── sub-02_scans.tsv
├── sub-03
│ ├── anat
│ │ ├── sub-03_T1w.json
│ │ └── sub-03_T1w.nii.gz
│ ├── func
│ │ ├── sub-03_task-con_run-01_bold.json
│ │ ├── sub-03_task-con_run-01_bold.nii.gz
│ │ ├── sub-03_task-con_run-01_events.tsv
│ │ ├── sub-03_task-con_run-02_bold.json
│ │ ├── sub-03_task-con_run-02_bold.nii.gz
│ │ ├── sub-03_task-con_run-02_events.tsv
│ │ ├── sub-03_task-con_run-03_bold.json
│ │ ├── sub-03_task-con_run-03_bold.nii.gz
│ │ ├── sub-03_task-con_run-03_events.tsv
│ │ ├── sub-03_task-con_run-04_bold.json
│ │ ├── sub-03_task-con_run-04_bold.nii.gz
│ │ ├── sub-03_task-con_run-04_events.tsv
│ │ ├── sub-03_task-con_run-05_bold.json
│ │ ├── sub-03_task-con_run-05_bold.nii.gz
│ │ ├── sub-03_task-con_run-05_events.tsv
│ │ ├── sub-03_task-con_run-06_bold.json
│ │ ├── sub-03_task-con_run-06_bold.nii.gz
│ │ ├── sub-03_task-con_run-06_events.tsv
│ │ ├── sub-03_task-con_run-07_bold.json
│ │ ├── sub-03_task-con_run-07_bold.nii.gz
│ │ ├── sub-03_task-con_run-07_events.tsv
│ │ ├── sub-03_task-con_run-08_bold.json
│ │ ├── sub-03_task-con_run-08_bold.nii.gz
│ │ ├── sub-03_task-con_run-08_events.tsv
│ │ ├── sub-03_task-loc_run-01_bold.json
│ │ ├── sub-03_task-loc_run-01_bold.nii.gz
│ │ ├── sub-03_task-loc_run-01_events.tsv
│ │ ├── sub-03_task-loc_run-02_bold.json
│ │ ├── sub-03_task-loc_run-02_bold.nii.gz
│ │ ├── sub-03_task-loc_run-02_events.tsv
│ │ ├── sub-03_task-ret_run-01_bold.json
│ │ ├── sub-03_task-ret_run-01_bold.nii.gz
│ │ ├── sub-03_task-ret_run-01_events.tsv
│ │ ├── sub-03_task-ret_run-02_bold.json
│ │ ├── sub-03_task-ret_run-02_bold.nii.gz
│ │ └── sub-03_task-ret_run-02_events.tsv
│ └── sub-03_scans.tsv
├── task-con_bold.json
├── task-loc_bold.json
└── task-ret_bold.json
`

My suspicion is that a weird “null byte” character is lurking in one of the JSON files created by heudiconv. https://stackoverflow.com/questions/18970830/how-to-find-null-byte-in-a-string-in-python

Ah, not just one. The key ‘DataSetTrailingPadding’ (which is inside the ‘slices’ key inside the ‘global’ key) is just a long list of null bytes. They’re listed as \u0000 when I look at them in a text editor but show up as \x00 when imported into python. Present in the .json that was generated for the anatomical and all of the functional images.

Does that key get used?

The key is not used, but all of the keys are hashed internally. That hashing process is thrown by the null bytes. Try running after removing this key.

just tried running fmriprep after deleting that key and it hasn’t crashed yet. So, that key seems to have been the issue. Thanks again for the help.

Is it true that there is no standard way of having heudiconv avoid generating that key in the first place (or, in general, avoiding generating a field that’s only going to be filled will null byte characters)?

You should try the --minmeta flag: https://github.com/nipy/heudiconv/blob/master/heudiconv/cli/run.py#L171

2 Likes

that does it! thanks.

1 Like

MANAGED BY INCF