Quick validation failed - the general folder structure does not resemble a BIDS dataset

RobertJirsaraie · September 6, 2018, 11:33pm

Hello BIDS Pipers,

I am running into an issue when trying to validate data for a single subject (Error Message Below). I believe that I am uploading the project folder correctly, but I cannot understand why bids is not reading my subject level directory. I have tried adding additional files (one more participant/participant.tsv/dataset_description.json) in the subject level directory but it has resulted in more errors (screenshot attached) or the website gets stuck on the validating page and does not produce any output.

I am hoping someone could help me understand what I may be doing wrong. I would be happy to email you the data I am working with or additional information.

Thank you,
Robert

Examples:
Path: mydataset/sub-001/ses-1/anat/sub-001_ses-1_T1w.nii.gz
Path: mydataset/sub-001/ses-1/anat/sub-001_ses-1_T1w.json

Error:
File Path: Quick validation failed - the general folder structure does not resemble a BIDS dataset. Have you chosen the right folder (with “sub-*/” subfolders)? Check for structural/naming issues and presence of at least one subject.

Type:		Error
File:		ses-1
Reason:		Quick validation failed - the general folder structure does not resemble a BIDS dataset. Have you chosen the right folder (with "sub-*/" subfolders)? Check for structural/naming issues and presence of at least one subject.

======================================================

ChrisGorgolewski · September 6, 2018, 11:43pm

The folder you should pass to the validator is the root of the dataset - in your example that would be mydataset/. This is also where the dataset_description.json file needs to be. So the minimal example will be

mydataset/dataset_description.json
mydataset/sub-001/ses-1/anat/sub-001_ses-1_T1w.nii.gz
mydataset/sub-001/ses-1/anat/sub-001_ses-1_T1w.json

It might be also useful to have a look at an example dataset such as this one: https://openneuro.org/datasets/ds000114/versions/1.0.0

RobertJirsaraie · September 6, 2018, 11:57pm

Hi Chris,

Thank you for your quick response. The minimal example that you provided is the exact way my files are structured and I just tried again to pass my root directory (mydataset) through the validator, but it seems to be stuck on the validating page.

ChrisGorgolewski · September 7, 2018, 12:03am

This could be a recent bug - could you try this version: https://1514-37161308-gh.circle-artifacts.com/0/root/web_version/index.html?

RobertJirsaraie · September 7, 2018, 12:14am

That version seemed to work. I still have two errors that are shown in the screenshot. I can problem solve the second error, but if you have any advise for the first error I would greatly appreciate it. Thank you very much Chris!

Capture

ChrisGorgolewski · September 7, 2018, 12:39am

If you click on the error box it will expand showing you which files are an issue. This should help resolving problems.

Example:
Before clicking

After clicking

jsein · December 11, 2018, 9:46pm

Hi, I think I found an error in the online version of the BIDS validator. Below is the error report I got. As you can see, there is an error because of the presence of the file “/event.json”. At the same time there are warnings due to “*_events.tsv” files without dictionary files associated to them. As a remedy, it is propose to add an JSON file with différent choice of naming… including “/events.json”!

Finally, when I created two files: “/task-localizer.json” and “/task-localizerMars.json” (corresponding to the two tasks present for this project), all the errors and warnings disappeared.

File Path: Files with such naming scheme are not part of BIDS specification. This error is most commonly caused by typos in file names that make them not BIDS compatible. Please consult the specification and make sure your files are named correctly. If this is not a file naming issue (for example when including files not yet covered by the BIDS specification) you should include a “.bidsignore” file in your dataset (see https://github.com/bids-standard/bids-validator#bidsignore for details). Please note that derived (processed) data should be placed in /derivatives folder and source data (such as DICOMS or behavioural logs in proprietary formats) should be placed in the /sourcedata folder.

Type:		Error
File:		events.json
Location:		RemiLocaMars/events.json
Reason:		Files with such naming scheme are not part of BIDS specification. This error is most commonly caused by typos in file names that make them not BIDS compatible. Please consult the specification and make sure your files are named correctly. If this is not a file naming issue (for example when including files not yet covered by the BIDS specification) you should include a ".bidsignore" file in your dataset (see https://github.com/bids-standard/bids-validator#bidsignore for details). Please note that derived (processed) data should be placed in /derivatives folder and source data (such as DICOMS or behavioural logs in proprietary formats) should be placed in the /sourcedata folder.
Evidence:	events.json

======================================================

File Path: Tabular file contains custom columns not described in a data dictionary

Type:		Warning
File:		sub-01_task-localizerMars_acq-mb_events.tsv
Location:		RemiLocaMars/sub-01/func/sub-01_task-localizerMars_acq-mb_events.tsv
Reason:		Tabular file contains custom columns not described in a data dictionary
Evidence:	Columns: expected_response, given_response not defined, please define in: /task-localizerMars_acq-mb_events.json, /task-localizerMars_events.json,/events.json,/sub-01/sub-01_task-localizerMars_acq-mb_events.json,/sub-01/sub-01_task-localizerMars_events.json,/sub-01/sub-01_events.json,/sub-01/func/sub-01_task-localizerMars_acq-mb_events.json,/sub-01/func/sub-01_task-localizerMars_events.json,/sub-01/func/sub-01_events.json

Type:		Warning
File:		sub-01_task-localizer_acq-mb_events.tsv
Location:		RemiLocaMars/sub-01/func/sub-01_task-localizer_acq-mb_events.tsv
Reason:		Tabular file contains custom columns not described in a data dictionary
Evidence:	Columns: expected_response, given_response not defined, please define in: /task-localizer_acq-mb_events.json, /task-localizer_events.json,/events.json,/sub-01/sub-01_task-localizer_acq-mb_events.json,/sub-01/sub-01_task-localizer_events.json,/sub-01/sub-01_events.json,/sub-01/func/sub-01_task-localizer_acq-mb_events.json,/sub-01/func/sub-01_task-localizer_events.json,/sub-01/func/sub-01_events.json

Type:		Warning
File:		sub-01_task-localizer_acq-sb_events.tsv
Location:		RemiLocaMars/sub-01/func/sub-01_task-localizer_acq-sb_events.tsv
Reason:		Tabular file contains custom columns not described in a data dictionary
Evidence:	Columns: expected_response, given_response not defined, please define in: /task-localizer_acq-sb_events.json, /task-localizer_events.json,/events.json,/sub-01/sub-01_task-localizer_acq-sb_events.json,/sub-01/sub-01_task-localizer_events.json,/sub-01/sub-01_events.json,/sub-01/func/sub-01_task-localizer_acq-sb_events.json,/sub-01/func/sub-01_task-localizer_events.json,/sub-01/func/sub-01_events.json

Type:		Warning
File:		sub-01_task-localizer_echo-1_events.tsv
Location:		RemiLocaMars/sub-01/func/sub-01_task-localizer_echo-1_events.tsv
Reason:		Tabular file contains custom columns not described in a data dictionary
Evidence:	Columns: expected_response, given_response not defined, please define in: /task-localizer_echo-1_events.json, /task-localizer_events.json,/events.json,/sub-01/sub-01_task-localizer_echo-1_events.json,/sub-01/sub-01_task-localizer_events.json,/sub-01/sub-01_events.json,/sub-01/func/sub-01_task-localizer_echo-1_events.json,/sub-01/func/sub-01_task-localizer_events.json,/sub-01/func/sub-01_events.json

Type:		Warning
File:		sub-01_task-localizer_echo-2_events.tsv
Location:		RemiLocaMars/sub-01/func/sub-01_task-localizer_echo-2_events.tsv
Reason:		Tabular file contains custom columns not described in a data dictionary
Evidence:	Columns: expected_response, given_response not defined, please define in: /task-localizer_echo-2_events.json, /task-localizer_events.json,/events.json,/sub-01/sub-01_task-localizer_echo-2_events.json,/sub-01/sub-01_task-localizer_events.json,/sub-01/sub-01_events.json,/sub-01/func/sub-01_task-localizer_echo-2_events.json,/sub-01/func/sub-01_task-localizer_events.json,/sub-01/func/sub-01_events.json

Type:		Warning
File:		sub-01_task-localizer_echo-3_events.tsv
Location:		RemiLocaMars/sub-01/func/sub-01_task-localizer_echo-3_events.tsv
Reason:		Tabular file contains custom columns not described in a data dictionary
Evidence:	Columns: expected_response, given_response not defined, please define in: /task-localizer_echo-3_events.json, /task-localizer_events.json,/events.json,/sub-01/sub-01_task-localizer_echo-3_events.json,/sub-01/sub-01_task-localizer_events.json,/sub-01/sub-01_events.json,/sub-01/func/sub-01_task-localizer_echo-3_events.json,/sub-01/func/sub-01_task-localizer_events.json,/sub-01/func/sub-01_events.json

======================================================

IvanSeleznov · January 11, 2019, 8:40am

Hi, I’m trying to check my dataset with online BIDS validator. Below is the error, which I’m not sure how to solve.
I have the dataset available on PhysioBank website (link to dataset) and I’m trying to make it BIDS like.
I’ve used the MNE python tool to create the BIDS directory and files structure (MNE tool). Below you can find the log of the tree cmd. I’ve tried to check fit the minimum dataset requirements and delete the README and other unnecessary files, but it’s also failing.

Could you, please, help me to cope with this error, as I’m not sure what it means?

ERROR:

File Path: Quick validation failed - the general folder structure does not resemble a BIDS dataset. Have you chosen the right folder (with “sub-*/” subfolders)? Check for structural/naming issues and presence of at least one subject.

Type: Error
File: eeg_mental
Reason: Quick validation failed - the general folder structure does not resemble a BIDS dataset. Have you chosen the right folder (with “sub-*/” subfolders)? Check for structural/naming issues and presence of at least one subject.

======================================================
File Path: Quick validation failed - the general folder structure does not resemble a BIDS dataset. Have you chosen the right folder (with “sub-*/” subfolders)? Check for structural/naming issues and presence of at least one subject.

Type: Error
File: eeg_mental
Reason: Quick validation failed - the general folder structure does not resemble a BIDS dataset. Have you chosen the right folder (with “sub-*/” subfolders)? Check for structural/naming issues and presence of at least one subject.

======================================================

DIRECTORY TREE: (I’ve truncated the directory tree to 4 subjects, as the same struct you can observe for all 35 participants)

eeg_mental/
├── dataset_description.json
├── participants.json
├── participants.tsv
├── README
├── sub-00
│ ├── eeg
│ │ ├── sub-00_task-counteyesclosed_channels.tsv
│ │ ├── sub-00_task-counteyesclosed_eeg.edf
│ │ ├── sub-00_task-counteyesclosed_eeg.json
│ │ ├── sub-00_task-counteyesclosed_events.tsv
│ │ ├── sub-00_task-resteyesclosed_channels.tsv
│ │ ├── sub-00_task-resteyesclosed_eeg.edf
│ │ ├── sub-00_task-resteyesclosed_eeg.json
│ │ └── sub-00_task-resteyesclosed_events.tsv
│ └── sub-00_scans.tsv
├── sub-01
│ ├── eeg
│ │ ├── sub-01_task-counteyesclosed_channels.tsv
│ │ ├── sub-01_task-counteyesclosed_eeg.edf
│ │ ├── sub-01_task-counteyesclosed_eeg.json
│ │ ├── sub-01_task-counteyesclosed_events.tsv
│ │ ├── sub-01_task-resteyesclosed_channels.tsv
│ │ ├── sub-01_task-resteyesclosed_eeg.edf
│ │ ├── sub-01_task-resteyesclosed_eeg.json
│ │ └── sub-01_task-resteyesclosed_events.tsv
│ └── sub-01_scans.tsv
├── sub-02
│ ├── eeg
│ │ ├── sub-02_task-counteyesclosed_channels.tsv
│ │ ├── sub-02_task-counteyesclosed_eeg.edf
│ │ ├── sub-02_task-counteyesclosed_eeg.json
│ │ ├── sub-02_task-counteyesclosed_events.tsv
│ │ ├── sub-02_task-resteyesclosed_channels.tsv
│ │ ├── sub-02_task-resteyesclosed_eeg.edf
│ │ ├── sub-02_task-resteyesclosed_eeg.json
│ │ └── sub-02_task-resteyesclosed_events.tsv
│ └── sub-02_scans.tsv
├── sub-03
│ ├── eeg
│ │ ├── sub-03_task-counteyesclosed_channels.tsv
│ │ ├── sub-03_task-counteyesclosed_eeg.edf
│ │ ├── sub-03_task-counteyesclosed_eeg.json
│ │ ├── sub-03_task-counteyesclosed_events.tsv
│ │ ├── sub-03_task-resteyesclosed_channels.tsv
│ │ ├── sub-03_task-resteyesclosed_eeg.edf
│ │ ├── sub-03_task-resteyesclosed_eeg.json
│ │ └── sub-03_task-resteyesclosed_events.tsv
│ └── sub-03_scans.tsv
├── sub-04
│ ├── eeg
│ │ ├── sub-04_task-counteyesclosed_channels.tsv
│ │ ├── sub-04_task-counteyesclosed_eeg.edf
│ │ ├── sub-04_task-counteyesclosed_eeg.json
│ │ ├── sub-04_task-counteyesclosed_events.tsv
│ │ ├── sub-04_task-resteyesclosed_channels.tsv
│ │ ├── sub-04_task-resteyesclosed_eeg.edf
│ │ ├── sub-04_task-resteyesclosed_eeg.json
│ │ └── sub-04_task-resteyesclosed_events.tsv
│ └── sub-04_scans.tsv

72 directories, 328 files

ChrisGorgolewski · January 11, 2019, 11:20am

EEG standardization is still in progress (EEG extension is still a proposal and has not been merged into the main spec). The online validator does not support EEG. You can use the command line version with the ‘–bep006’ flag to validate EEG datasets.

dvanblooijs · September 18, 2020, 12:47pm

Hi all,
i’m experiencing the same error jsein is writing about in Dec '18. Would you recommend to replace events.json by a task-localizer.json for example?

dvanblooijs · September 22, 2020, 9:15am

I tried to rename the events.json to task-X.json, but this did not help in solving the error…

franklin · September 22, 2020, 8:53pm

Hi @dvanblooijs

Welcome to NeuroStars and thank you for your message! Perhaps may you please describe a bit more your case?

Thank you,
Franklin

dvanblooijs · September 23, 2020, 7:45am

Hi @franklin,

thank you for your response! I have six subjects with several tasks, but all these different tasks result in the same columns in each subject’s events.tsv. I also made one events.json which describe the extra columns in events.tsv and located that in dataset/events.json. All subject’s events.tsv (and corresponding ecogs, channels.tsv etc.) are located in dataset/sub-001/ses-1/ieeg/sub-001_ses-1_task-X_events.tsv. As I read from jsein, he changed the name of the events.json to task-X.json, which helped in fixing the error the bids validator gave:

events.json 1.153 KB | application/json

Location:

dataset descriptor/events.json

Reason:

Files with such naming scheme are not part of BIDS specification. This error is most commonly caused by typos in file names that make them not BIDS compatible. Please consult the specification and make sure your files are named correctly. If this is not a file naming issue (for example when including files not yet covered by the BIDS specification) you should include a “.bidsignore” file in your dataset (see https://github.com/bids-standard/bids-validator#bidsignore for details). Please note that derived (processed) data should be placed in /derivatives folder and source data (such as DICOMS or behavioural logs in proprietary formats) should be placed in the /sourcedata folder.

Evidence:

events.json

At the same time, I get this warning:
sub-RESP0521_ses-1_task-SPESclin_run-021231_events.tsv 80.253 KB |

Location:

dataset descriptor/sub-RESP0521/ses-1/ieeg/sub-RESP0521_ses-1_task-SPESclin_run-021231_events.tsv

Reason:

Tabular file contains custom columns not described in a data dictionary

Evidence:

Columns: sub_type, electrodes_involved_onset, electrodes_involved_offset, offset, sample_start, sample_end, electrical_stimulation_type, electrical_stimulation_site, electrical_stimulation_current, electrical_stimulation_frequency, electrical_stimulation_pulsewidth, notes not defined, please define in: /sub-RESP0521/ses-1/ieeg/sub-RESP0521_ses-1_events.json, /sub-RESP0521/ses-1/ieeg/sub-RESP0521_ses-1_task-SPESclin_events.json,/sub-RESP0521/ses-1/sub-RESP0521_ses-1_events.json,/sub-RESP0521/ses-1/sub-RESP0521_ses-1_task-SPESclin_events.json,/sub-RESP0521/sub-RESP0521_events.json,/sub-RESP0521/sub-RESP0521_task-SPESclin_events.json,/events.json,/task-SPESclin_events.json,/sub-RESP0521/ses-1/ieeg/sub-RESP0521_ses-1_task-SPESclin_run-021231_events.json

Apparently, the events.json is not recognized as a descriptor of the colums in the events.tsv files. I get similar errors for the scans.json and the electrodes.json.

Dorien

dvanblooijs · September 23, 2020, 4:39pm

If I name the events.json task-X_events.json, the error disappears. However, since the events.json is equal for any kind of task, I prefer using events.json instead of task-X_events.json for each kind of event.

For electrodes.json and scans.json, there is no specification for a task, so the solution above is not possible for these files. I hope you can help me out!

franklin · September 23, 2020, 5:54pm

hi @dvanblooijs

Thank you for the additional information. Addressing your points:

I believe an events file needs to be associated with a task. This would support the case seen by you and jsein.
The second error is arising because of the events.json not being seen (from the first error). Once the events.json is resolved this will too.
Regarding electrodes, they are defined with a *_coordsystem.json file. I presume this would hold up to a subject level rather than at root.
Regarding scans, this is utilized at the session level to describe the acquisitions taken.

dvanblooijs · September 24, 2020, 7:18am

Hi @franklin,

Thank you for your response! In the Evidence of the events.json error I posted, one of the options is to make an /events.json (I made it bold in my previous post showing the error).

The same applies to the evidence in the electrodes.json error. I will post it next, as well as the scans.json error.

I think that locating these jsonfiles at subject level would solve these issues, but that would lead to multiple equal descriptive files in all subject folders. It would be much easier, and less prone to errors to save these files once at a higher level. The evidence of the errors seem to support this higher level location, but maybe that it is a mistake in the description of the error?

Thanks for your help!

Small update: putting electrodes.json and scans.json at subject level did solve this issue, so at least my dataset is now bids compatible!

dvanblooijs · September 24, 2020, 7:21am

sub-RESP0521_ses-1_electrodes.tsv 17.124 KB |

Location:

dataset descriptor/sub-RESP0521/ses-1/ieeg/sub-RESP0521_ses-1_electrodes.tsv

Reason:

Tabular file contains custom columns not described in a data dictionary

Evidence:

Columns: silicon, soz, resected not defined, please define in: /electrodes.json, /sub-RESP0521/sub-RESP0521_electrodes.json,/sub-RESP0521/ses-1/sub-RESP0521_ses-1_electrodes.json,/sub-RESP0521/ses-1/ieeg/sub-RESP0521_ses-1_electrodes.json

dvanblooijs · September 24, 2020, 7:22am

sub-RESP0521_ses-1_scans.tsv 0.635 KB |

Location:

datasetDescriptor/sub-RESP0521/ses-1/sub-RESP0521_ses-1_scans.tsv

Reason:

Tabular file contains custom columns not described in a data dictionary

Evidence:

Columns: format, artefact not defined, please define in: /scans.json, /sub-RESP0521/sub-RESP0521_scans.json,/sub-RESP0521/ses-1/sub-RESP0521_ses-1_scans.json

franklin · September 24, 2020, 5:03pm

hi @dvanblooijs

Great to hear the dataset is now BIDS validated!