Dataset_description.json missing from project root, file exists in project root

Software Support
1

Hello,

Summary of what happened:

I am running QSIPrep on MRI data, using a docker command. When I execute the command, I get an error saying File “/usr/local/miniconda/lib/python3.8/site-packages/bids/layout/validation.py”, line 75, in validate_root
raise BIDSValidationError(
bids.exceptions.BIDSValidationError: ‘dataset_description.json’ is missing from project root.

The file dataset_description.json is in the file that is used as the working directory, validated by typing pwd in the terminal.

Command used (and if a helper script was used, a link to the helper script or the command generated):

subj_id=$602

# Introduce the BIDS directory
bids_dir=/Users/Dutcher_Imaging/Desktop/headspace/HeadspaceData
# Define the output directory for the input subject
output_dir=/Users/Dutcher_Imaging/Desktop/headspace/HeadspaceData/sub-6$subj_id/output
# Create folder if it does not exist
sudo mkdir -p $output_dir
work_dir=$output_dir/work
sudo mkdir -p $work_dir
freesurfer_home=/Users/Dutcher_Imaging/Desktop/SourceCode/qsiprep_build-main
fs_license=$freesurfer_home/freesurfer_license.txts

docker_cmd="docker run --rm \
    -v ${bids_dir}:/data \
    -v ${output_dir}:/output \
    -v ${work_dir}:/work \
    -v ${fs_license}:/fs_license.txt \
    pennbbl/qsiprep:0.19.1"
cmd="${docker_cmd}  /data /output participant\
    --skip-bids-validation \
    --ignore fieldmaps \
    -w /work \
    --participant_label ${subj_id} \
    --output-resolution 2 \
    --unringing-method mrdegibbs \
    --n_cpus 5 \
    --fs-license-file /fs_license.txt"

eval $cmd

Version:

0.19.1

Environment (Docker, Singularity / Apptainer, custom installation):

docker

Data formatted according to a validatable standard? Please provide the output of the validator:

bids-standard.github.io/bids-validator/ returns only fieldmap errors, which can be ignored

Relevant log outputs (up to 20 lines):

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/local/miniconda/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/cli/run.py", line 950, in build_qsiprep_workflow
    retval['workflow'] = init_qsiprep_wf(
  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/workflows/base.py", line 228, in init_qsiprep_wf
    single_subject_wf = init_single_subject_wf(
  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/workflows/base.py", line 481, in init_single_subject_wf
    subject_data, layout = collect_data(bids_dir, subject_id, filters=bids_filters)
  File "/usr/local/miniconda/lib/python3.8/site-packages/qsiprep/utils/bids.py", line 145, in collect_data
    layout = BIDSLayout(str(bids_dir), validate=bids_validate)
  File "/usr/local/miniconda/lib/python3.8/site-packages/bids/layout/layout.py", line 135, in __init__
    root, description = validate_root(root, validate)
  File "/usr/local/miniconda/lib/python3.8/site-packages/bids/layout/validation.py", line 75, in validate_root
    raise BIDSValidationError(
bids.exceptions.BIDSValidationError: 'dataset_description.json' is missing from project root. Every valid BIDS dataset must have this file.
Example contents of 'dataset_description.json':
{"Name": "Example dataset", "BIDSVersion": "1.0.2"}

Screenshots / relevant information:

Thank you so much, any help is much appreciated!
-Angelica

2

Hi @acrown and welcome to neurostars!

Is dataset_desciption.json in your bids_dir (/Users/Dutcher_Imaging/Desktop/headspace/HeadspaceData)?

Best,
Steven

3

Hi Steven,

Thank you for your quick response!

Yes, the dataset_description is in the HeadspaceData folder but the error still pops up every time.

Thanks!
Angelica

4

Hi @acrown,

Can you use the tree command to return the directory structure of your BIDS directory?

Best,
Steven

5

Hi Steven,

Yes, here is the output of the tree command.


.
├── qsiprep
│   ├── sub-01
│   │   ├── qsiprep
│   │   │   └── logs
│   │   └── work
│   │       ├── 20240620-185748_dea9c7ea-0512-46e5-b476-149c423d3a6e
│   │       │   └── bids_db
│   │       │       └── layout_index.sqlite
│   │       ├── 20240620-185955_0f5dff0e-7b4b-482e-9367-06836e5f33ea
│   │       │   └── bids_db
│   │       │       └── layout_index.sqlite
│   │       ├── 20240620-190410_77424f01-f3b8-4e5a-873b-e642c23d1637
│   │       │   └── bids_db
│   │       │       └── layout_index.sqlite
│   │       ├── 20240620-190747_82d0cf4a-3d4a-41c5-a637-0bffc956f506
│   │       │   └── bids_db
│   │       │       └── layout_index.sqlite
│   │       ├── 20240621-160317_a65e9e68-a30f-4e06-935d-079162297421
│   │       │   └── bids_db
│   │       │       └── layout_index.sqlite
│   │       ├── 20240621-175351_2a9814b3-6798-49ec-bdf7-90d0313459fc
│   │       │   └── bids_db
│   │       │       └── layout_index.sqlite
│   │       ├── 20240621-180018_5e5a1d32-b8b9-40d7-93c5-65350af0ca26
│   │       │   └── bids_db
│   │       │       └── layout_index.sqlite
│   │       ├── 20240621-180114_7806fdc9-3d1f-4118-b47f-d38f519f5535
│   │       │   └── bids_db
│   │       │       └── layout_index.sqlite
│   │       └── 20240621-180526_96befb23-53cf-4470-a067-7430ba97335e
│   │           └── bids_db
│   │               └── layout_index.sqlite
│   ├── sub-01s1
│   │   ├── qsiprep
│   │   │   └── logs
│   │   └── work
│   │       ├── 20240620-191156_9c10f6a7-54f5-4c12-90b8-4414b287c448
│   │       │   └── bids_db
│   │       │       └── layout_index.sqlite
│   │       └── 20240620-191305_ca8f822f-c4ba-421b-a25a-553131c021c9
│   │           └── bids_db
│   │               └── layout_index.sqlite
│   └── sub-02
│       ├── output
│       │   ├── qsiprep
│       │   │   └── logs
│       │   └── work
│       │       ├── 20240621-183310_da194813-b444-425c-af78-5c23aadd4556
│       │       │   └── bids_db
│       │       ├── 20240621-183448_dcf98e8c-40fc-48b5-b2d9-e1c47a0fc018
│       │       │   └── bids_db
│       │       └── 20240624-154657_0265529e-51cf-46d6-843c-7919f294f979
│       │           └── bids_db
│       │               └── layout_index.sqlite
│       ├── qsiprep
│       │   └── logs
│       └── work
│           ├── 20240621-181323_ebfc8d9c-9811-4085-9a76-2e7783f2dee9
│           │   └── bids_db
│           │       └── layout_index.sqlite
│           ├── 20240621-181733_34d2fd56-f860-4ce8-a1cf-c9353d555f0c
│           │   └── bids_db
│           │       └── layout_index.sqlite
│           ├── 20240621-182158_2416cd6e-ce24-4f9e-9388-008d9e394487
│           │   └── bids_db
│           │       └── layout_index.sqlite
│           └── 20240621-182524_0f2b1a77-6973-4f01-87d9-ad07cbd899d3
│               └── bids_db
│                   └── layout_index.sqlite
├── sub-02
│   └── output
│       └── work
│           └── 20240624-174609_99722aa7-421a-479a-a42b-577dd0097734
│               └── bids_db
│                   └── layout_index.sqlite
└── sub-602
    └── output
        ├── qsiprep
        │   └── logs
        └── work
            ├── 20240624-175848_d07c14a2-34e0-4f9b-96a6-2b33ca4b379f
            │   └── bids_db
            │       └── layout_index.sqlite
            └── 20240624-181152_0d5222c0-e87f-4997-a41a-6f69b27b9c0e
                └── bids_db
                    └── layout_index.sqlite

67 directories, 19 files

Thank you!
Angelica

6

Hi @acrown,

That doesn’t look like the raw BIDS root directory. It should contain the sub-XX folders with the raw data.

Best,
Steven

7

Hi @Steven

There is some issue with how the script is reading the data. On the local drive, the HeadspaceData is a valid bids dataset, validated by https://bids-standard.github.io/bids-validator/ and includes the participants.tsv and dataset_description.json files, as well as folders with the raw data for 100 participants. The working directory (manually set using the cd command) points directly to this HeadspaceData folder (pwd outputs the correct folder name and path (/Users/Dutcher_Imaging/Desktop/headspace/HeadspaceData)) but in the docker command (and the tree command) something gets messed up. I am pretty sure that my bids_dir, output_dir, and work_dir are pointing to the right place, but it seems like they might not be due to these errors.

Code for directories - also in first post

#Introduce the BIDS directory
bids_dir=/Users/Dutcher_Imaging/Desktop/headspace/HeadspaceData
#Define the output directory for the input subject
output_dir=/Users/Dutcher_Imaging/Desktop/headspace/HeadspaceData/sub-6$subj_id/output
#Create working directory for intermdiate results. This directory is deleted after qsiprep completion.
work_dir=$output_dir/work

Folder on local computer structure

files
files1043×1013 61.3 KB

If you have any ideas on what the issue is, any help is much appreciated.

Thank you!
Angelica

8

inside sub-XX
inside sub-XX1104×804 28.4 KB

9

inside func
inside func1098×804 34.9 KB

10

Output dir should be /Users/Dutcher_Imaging/Desktop/headspace/HeadspaceData/derivatives. What you are currently doing is putting their preprocessed data in the raw folders, which is not good.

I wouldn’t put work dir in the output dir. Output directories are also supposed to be BIDS valid sets themselves.

11

Hi @Steven

Thank you! Where would you suggest putting the working directory?

Thank you for your help, it is greatly appreciated.
-Angelica

12

Also @Steven,

I changed the directories to what you suggested, and the work_dir to /Users/Dutcher_Imaging/Desktop/headspace/HeadspaceData/work but the dataset_description.json missing from project root error still persists