ValueError: BIDS root does not exist

Lara · July 14, 2019, 7:05am

Dear experts,

I am new to neuroimaging, and need some help with fMRIPrep.

The following errors are received when running my fmriprep-docker command:

Traceback (most recent call last):

File “/usr/local/miniconda/bin/fmriprep”, line 10, in *
sys.exit(main())*
File “/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/cli/run.py”, line 414, in main*
subject_list, output_dir, work_dir, run_uuid, packagename=‘fmriprep’)*
File “/usr/local/miniconda/lib/python3.7/site-packages/niworkflows/reports/core.py”, line 420, in generate_reports*
for subject_label in subject_list*
File “/usr/local/miniconda/lib/python3.7/site-packages/niworkflows/reports/core.py”, line 420, in *
for subject_label in subject_list*
File “/usr/local/miniconda/lib/python3.7/site-packages/niworkflows/reports/core.py”, line 407, in run_reports*
subject_id=subject_label, packagename=packagename)*
File “/usr/local/miniconda/lib/python3.7/site-packages/niworkflows/reports/core.py”, line 256, in init*
self._load_config(Path(config))*
File “/usr/local/miniconda/lib/python3.7/site-packages/niworkflows/reports/core.py”, line 274, in _load_config*
self.index(settings[‘sections’])*
File “/usr/local/miniconda/lib/python3.7/site-packages/niworkflows/reports/core.py”, line 283, in index*
self.layout = BIDSLayout(self.root, config=‘figures’, validate=False)*
File “/usr/local/miniconda/lib/python3.7/site-packages/bids/layout/layout.py”, line 146, in init*
raise ValueError(“BIDS root does not exist: %s” % root)*
ValueError: BIDS root does not exist: /tmp/work/reportlets/fmriprep/sub-13
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File “/usr/local/miniconda/lib/python3.7/concurrent/futures/process.py”, line 101, in _python_exit*
thread_wakeup.wakeup()*
File “/usr/local/miniconda/lib/python3.7/concurrent/futures/process.py”, line 89, in wakeup*
self._writer.send_bytes(b"")*
File “/usr/local/miniconda/lib/python3.7/multiprocessing/connection.py”, line 183, in send_bytes*
self._check_closed()*
File “/usr/local/miniconda/lib/python3.7/multiprocessing/connection.py”, line 136, in _check_closed*
raise OSError(“handle is closed”)*
OSError: handle is closed
Sentry is attempting to send 1 pending error messages

Also, the “fig” directory is not created. Only the “anat” and “func” directories are there.

Does anyone have any ideas what I am doing wrong?

Kind regards
Lara

PeerHerholz · July 14, 2019, 6:52pm

Ahoi hoi @Lara,

welcome to Neurostars, it’s great to have you here!

Cool thing you’re using fMRIPrep. Could you maybe provide
further information on your setup, e.g. OS and fMRIPrep version, as well
as what you’re trying to do? Maybe also include the original command you run? With that it’s way easier for the community to help you.

Best, cheers, Peer

Lara · July 15, 2019, 10:40am

Hi Peer

Great to chat with you.
I am using fMRIPrep 1.4.0, OS Ubuntu 16.04, (64-bit), 8GB Ram
I need to run preprocessing for dual regression analysis, I am a bit concerned because when I run my command:

fmriprep-docker --participant_label=sub-01 -vvv --fs-license-file /usr/local/freesurfer/license.txt --use-aroma --write-graph --stop-on-first-crash --ignore=fieldmaps /home/Session2/Input/ /home/Session2/Output participant

I immediately get a notice saying that I may have insufficient RAM, but I pushed past this. Do you think this may be the problem?

Cheers
Lara

effigies · July 15, 2019, 2:11pm

ValueError: BIDS root does not exist: /tmp/work/reportlets/fmriprep/sub-13

This seems like an odd place to be looking, and it’s unclear why sub-13 should appear anywhere if you use --participant-label=sub-01 (by the way, the label should be 01, not sub-01…). When you run fmriprep-docker, the first thing to appear will be a line that starts with RUNNING:. Could you provide that? And if you have a full log, that might be useful.

It also might be helpful to use the -w flag to use a working directory external to the container. The working directory can be very large, and can sometimes fill up the space Docker allocates for files inside the container, so using -w will mount a working directory.

PeerHerholz · July 15, 2019, 2:48pm

what @effigies said.

Also, you might want to rearrange the arguments of the command to:

fmriprep-docker -vvv --fs-license-file /usr/local/freesurfer/license.txt /home/Session2/Input/ /home/Session2/Output -w /home/Session2/workdir participant --participant_label 01 --use-aroma --write-graph --stop-on-first-crash --ignore=fieldmaps

Lara · July 16, 2019, 7:19am

Hi @effigies and Peer

Thanks for the feedback.
Unfortunately I don’t have the entire log (it doesn’t give me all of the history), but below is the initiating log (first line etc.)

I have also modified my bash script and will try that today.

Kind regards
Lara

RUNNING: docker run --rm -it -e DOCKER_VERSION_8395080871=18.09.7 -v /usr/local/freesurfer/license.txt:/opt/freesurfer/license.txt:ro -v /home/lara/Documents/Lara/CM/cm_mri/Session2/Input:/data:ro -v /home/lara/Documents/Lara/CM/cm_mri/Session2/Output:/out poldracklab/fmriprep:1.4.0 /data /out participant --participant_label=sub-15 -vvv --use-aroma --write-graph --stop-on-first-crash --ignore=fieldmaps
Making sure the input data is BIDS compliant (warnings can be ignored in most cases).
1: [ERR] Files with such naming scheme are not part of BIDS specification. This error is most commonly caused by typos in file names that make them not BIDS compatible. Please consult the specification and make sure your files are named correctly. If this is not a file naming issue (for example when including files not yet covered by the BIDS specification) you should include a “.bidsignore” file in your dataset (see https://github.com/bids-standard/bids-validator#bidsignore for details). Please note that derived (processed) data should be placed in /derivatives folder and source data (such as DICOMS or behavioural logs in proprietary formats) should be placed in the /sourcedata folder. (code: 1 - NOT_INCLUDED)
./sub-15/session2/anat/sub-69_T1w.json
Evidence: sub-69_T1w.json
./sub-15/session2/anat/sub-69_T1w.nii.gz
Evidence: sub-69_T1w.nii.gz
./sub-15/session2/fmap/sub-69_magnitude1.nii.gz
Evidence: sub-69_magnitude1.nii.gz
./sub-15/session2/fmap/sub-69_phasediff.json
Evidence: sub-69_phasediff.json
./sub-15/session2/fmap/sub-69_phasediff.nii.gz
Evidence: sub-69_phasediff.nii.gz
./sub-15/session2/func/sub-69_task-rest_bold.json
Evidence: sub-69_task-rest_bold.json
./sub-15/session2/func/sub-69_task-rest_bold.nii.gz
Evidence: sub-69_task-rest_bold.nii.gz

Please visit https://neurostars.org/search?q=NOT_INCLUDED for existing conversations about this issue.

2: [ERR] Dataset does not contain any T1w scans. (code: 53 - NO_T1W)

Please visit https://neurostars.org/search?q=NO_T1W for existing conversations about this issue.

3: [ERR] Subject label in the filename doesn't match with the path of the file. File seems to be saved in incorrect subject directory. (code: 64 - SUBJECT_LABEL_IN_FILENAME_DOESNOT_MATCH_DIRECTORY)
	./sub-15/session2/anat/sub-69_T1w.json
		Evidence: File: /sub-15/session2/anat/sub-69_T1w.json is saved in incorrect subject directory as per sub-id in filename.
	./sub-15/session2/anat/sub-69_T1w.nii.gz
		Evidence: File: /sub-15/session2/anat/sub-69_T1w.nii.gz is saved in incorrect subject directory as per sub-id in filename.
	./sub-15/session2/fmap/sub-69_magnitude1.nii.gz
		Evidence: File: /sub-15/session2/fmap/sub-69_magnitude1.nii.gz is saved in incorrect subject directory as per sub-id in filename.
	./sub-15/session2/fmap/sub-69_phasediff.json
		Evidence: File: /sub-15/session2/fmap/sub-69_phasediff.json is saved in incorrect subject directory as per sub-id in filename.
	./sub-15/session2/fmap/sub-69_phasediff.nii.gz
		Evidence: File: /sub-15/session2/fmap/sub-69_phasediff.nii.gz is saved in incorrect subject directory as per sub-id in filename.
	./sub-15/session2/func/sub-69_task-rest_bold.json
		Evidence: File: /sub-15/session2/func/sub-69_task-rest_bold.json is saved in incorrect subject directory as per sub-id in filename.
	./sub-15/session2/func/sub-69_task-rest_bold.nii.gz
		Evidence: File: /sub-15/session2/func/sub-69_task-rest_bold.nii.gz is saved in incorrect subject directory as per sub-id in filename.

Please visit https://neurostars.org/search?q=SUBJECT_LABEL_IN_FILENAME_DOESNOT_MATCH_DIRECTORY for existing conversations about this issue.

4: [ERR] No BIDS compatible data found for at least one subject. (code: 67 - NO_VALID_DATA_FOUND_FOR_SUBJECT)
	./sub-02
	./sub-03
	./sub-04
	./sub-05
	./sub-06
	./sub-07
	./sub-09
	./sub-10
	./sub-12
	./sub-13
	... and 19 more files having this issue (Use --verbose to see them all).

Please visit https://neurostars.org/search?q=NO_VALID_DATA_FOUND_FOR_SUBJECT for existing conversations about this issue.

1: [WARN] The recommended file /README is missing. See Section 03 (Modality agnostic files) of the BIDS specification. (code: 101 - README_FILE_MISSING)

Please visit https://neurostars.org/search?q=README_FILE_MISSING for existing conversations about this issue.


    Summary:                Available Tasks:        Available Modalities: 
    401 Files, 1.5GB                                                      
    0 - Subjects                                                          
    1 - Session                                                           


If you have any questions, please post on https://neurostars.org/tags/bids.

^CProcess Process-2:
Traceback (most recent call last):
File “/usr/local/miniconda/lib/python3.7/multiprocessing/process.py”, line 297, in _bootstrap
self.run()
File “/usr/local/miniconda/lib/python3.7/multiprocessing/process.py”, line 99, in run
self._target(*self._args, **self._kwargs)
File “/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/cli/run.py”, line 445, in build_workflow
from …workflows.base import init_fmriprep_wf
File “/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/workflows/base.py”, line 25, in
from niworkflows.interfaces.bids import (
File “/usr/local/miniconda/lib/python3.7/site-packages/niworkflows/interfaces/init.py”, line 6, in
from .confounds import ExpandModel, SpikeRegressors
File “/usr/local/miniconda/lib/python3.7/site-packages/niworkflows/interfaces/confounds.py”, line 11, in
import pandas as pd
File “/usr/local/miniconda/lib/python3.7/site-packages/pandas/init.py”, line 42, in
from pandas.core.api import *
File “/usr/local/miniconda/lib/python3.7/site-packages/pandas/core/api.py”, line 10, in
from pandas.core.groupby.groupby import Grouper
File “/usr/local/miniconda/lib/python3.7/site-packages/pandas/core/groupby/init.py”, line 2, in
from pandas.core.groupby.groupby import (
File “/usr/local/miniconda/lib/python3.7/site-packages/pandas/core/groupby/groupby.py”, line 46, in
from pandas.core.index import (Index, MultiIndex,
File “/usr/local/miniconda/lib/python3.7/site-packages/pandas/core/index.py”, line 2, in
from pandas.core.indexes.api import *
File “/usr/local/miniconda/lib/python3.7/site-packages/pandas/core/indexes/api.py”, line 4, in
from pandas.core.indexes.base import (Index,
File “”, line 983, in _find_and_load
File “”, line 967, in _find_and_load_unlocked
File “”, line 677, in _load_unlocked
File “”, line 724, in exec_module
File “”, line 857, in get_code
File “”, line 525, in _compile_bytecode
KeyboardInterrupt
Traceback (most recent call last):
File “/usr/local/miniconda/bin/fmriprep”, line 10, in
sys.exit(main())
File “/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/cli/run.py”, line 340, in main
p.join()
File “/usr/local/miniconda/lib/python3.7/multiprocessing/process.py”, line 140, in join
res = self._popen.wait(timeout)
File “/usr/local/miniconda/lib/python3.7/multiprocessing/popen_fork.py”, line 48, in wait
return self.poll(os.WNOHANG if timeout == 0.0 else 0)
File “/usr/local/miniconda/lib/python3.7/multiprocessing/popen_forkserver.py”, line 61, in poll
if not wait([self.sentinel], timeout):
File “/usr/local/miniconda/lib/python3.7/multiprocessing/connection.py”, line 920, in wait
ready = selector.select(timeout)
File “/usr/local/miniconda/lib/python3.7/selectors.py”, line 415, in select
fd_event_list = self._selector.poll(timeout)
KeyboardInterrupt
Sentry is attempting to send 0 pending error messages
Waiting up to 2.0 seconds
Press Ctrl-C to quit
fMRIPrep: Please report errors to https://github.com/poldracklab/fmriprep/issues

effigies · July 16, 2019, 1:44pm

The validator errors indicate that your dataset is not formatted as a BIDS dataset. The obvious thing that sticks out to me is the directory called session2 instead of ses-2, or similar.

Lara · July 17, 2019, 7:04am

Session2 is a folder I created that holds my BIDS data.
Thank you for your help though

effigies · July 17, 2019, 12:41pm

I meant session2 here:

Lara · July 21, 2019, 7:36am

Hi
I found the problem.
In the last line you sent there are 2 separate names for the same subject. I renamed them to match and removed that second session2 directory. Working like a dream now.

Thanks for the help.
Cheers
Lara