I am organizing an old study into BIDS. When I run the BIDS Validator, it is returning errors saying fields are missing that are not.
My BIDS directory contains the following files: dataset_description.json, README, task-avoid_events.json, and task-faces_events.json. It also contains one subject directory: sub-MWMH378.
Within the subject directory, there is a session directory that contains anat, func, and dwi directories. Within the func directory, there are the following files:
- sub-MWMH378_ses-1_task-avoid_bold.json
- sub-MWMH378_ses-1_task-avoid_bold.nii.gz
- sub-MWMH378_ses-1_task-avoid_events.tsv
- sub-MWMH378_ses-1_task-faces_bold.json
- sub-MWMH378_ses-1_task-faces_bold.nii.gz
- sub-MWMH378_ses-1_task-faces_events.tsv
- sub-MWMH378_ses-1_task-rest_bold.json
- sub-MWMH378_ses-1_task-rest_bold.nii.gz
The BIDS validator is returning the following errors:
1: [ERR] First column of the events file must be named 'onset' (code: 20 - EVENTS_COLUMN_ONSET)
2: [ERR] Second column of the events file must be named 'duration' (code: 21 - EVENTS_COLUMN_DURATION), and
3: [ERR] You have to define 'TaskName' for this file. (code: 50 - TASK_NAME_MUST_DEFINE)
And the following warnings:
1: [WARN] Tabular file contains custom columns not described in a data dictionary (code: 82 - CUSTOM_COLUMN_WITHOUT_DESCRIPTION)
Errors
1 & 2.) In my events files (sub-MWMH378_ses-1_task-avoid_events.tsv and sub-MWMH378_ses-1_task-faces_events.tsv), the columns are as follows:
For avoid: “onset” “duration” “trial” “cue” “fix1” “feedback” “fix2” “approach” “avoid” “reward” “loss” “nothing” “gain50” “gain10” “lose10” “lose50”
For faces: “onset” “duration” “trial” “face” “blank” “fix” “female” “happy” “intensity10” “intensity20” “intensity30” “intensity40” “intensity50” “correct” “press”
So both start with “onset” and have “duration” second. Why is the BIDS Validator throwing an error here?
3.) The task field is specified in all of the functional file names. Is this error about some other field? If so, where?
Warnings
- I have jsons for both of the tasks in the bids directory that define all of the columns. Maybe my files are named incorrectly, so they aren’t being found? I’d appreciate any insight here. Example column definition:
“cue”: {
“LongName”: “Cue item”,
“Description”: “Whether or not this is a cue item”,
“Levels”: {
“0”: “Not a cue item”,
“1”: “A cue item”
}
Thank you all for your help!
I believe the problem is the trial field. It should be trial_type.
onset duration trial_type cue fix1 ....
ref
I just double checked the BIDS specification, and it looks like trial_type is an optional column. I am using trial here for the trial number, not to define the nature of the trial. Why might using trial as a column cause issues with the BIDS validator not recognizing that onset and duration are the first and second columns, respectively?
Yes, good point.
I just modified one of my _events.tsv files and used a space instead of tab between onset duration trial_type and got the error you received. So my suspicion is that your file is not tab separated, but has spaces within.
As you can see, it looks okay:
(nipype) Johns-iMac-Pro :: derivatives/output/BIDS 1 » head sub-01/func/sub-01_task-ao_run-01_events.tsv
onset duration trial_type
13 12 fixation
31 8 intact_alt
39 8 fixation
...
But here’s some of the error text:
(nipype) Johns-iMac-Pro :: derivatives/output/BIDS » bids-validator .
bids-validator@1.9.3
bids-specification@disable
(node:94633) Warning: Closing directory handle on garbage collection
(Use `node --trace-warnings ...` to show where the warning was created)
1: [ERR] First column of the events file must be named 'onset' (code: 20 - EVENTS_COLUMN_ONSET)
./sub-01/func/sub-01_task-ao_run-01_events.tsv
@ line: 1
Evidence: Column headers: onset duration trial_type
Please visit https://neurostars.org/search?q=EVENTS_COLUMN_ONSET for existing conversations about this issue.
2: [ERR] Second column of the events file must be named 'duration' (code: 21 - EVENTS_COLUMN_DURATION)
./sub-01/func/sub-01_task-ao_run-01_events.tsv
@ line: 1
Evidence: Column headers: onset duration trial_type
Please visit https://neurostars.org/search?q=EVENTS_COLUMN_DURATION for existing conversations about this issue.
Oh, interesting. Thank you for looking into this! I am creating the _events.tsv files with R. Specifically, with the following call:
write.table(final_faces_df, paste0(bids_path, sub, '/ses-', ses, '/func/sub-MWMH', sub, '_ses-', ses, '_task-faces_events.tsv'), row.names=FALSE, sep='\t')
Do you know if there is another function I should be using?
I just tried it with some of my data, and it works properly. After I turned on Show invisible characters, you can see in my editor (Textmate) that the little tab characters are there, rather than spaces.
I would look at the .tsv file in an editor and try to see what’s going on.
If you’re comfortable in a terminal, you could use hexdump to look at the contents. You’re looking for the code 09, which is a tab character. Space is 20.
(base) Johns-iMac-Pro :: Data/Research/ignored_stuff » hexdump -C test_outfile.tsv | head
00000000 6d 6f 6e 74 61 67 65 09 66 72 65 71 5f 72 61 6e |montage.freq_ran|
00000010 67 65 09 63 68 61 6e 6e 65 6c 73 09 73 75 62 6a |ge.channels.subj|
00000020 65 63 74 09 63 6f 6e 64 69 74 69 6f 6e 09 77 68 |ect.condition.wh|
00000030 69 63 68 5f 68 61 6c 66 09 65 70 6f 63 68 09 6d |ich_half.epoch.m|
00000040 6f 76 65 09 70 68 61 73 65 09 6d 65 61 6e 5f 70 |ove.phase.mean_p|
00000050 6f 77 65 72 5f 61 62 73 0a 46 72 6f 6e 74 61 6c |ower_abs.Frontal|
00000060 09 34 48 7a 2d 37 48 7a 09 46 31 20 46 5a 09 31 |.4Hz-7Hz.F1 FZ.1|
00000070 38 09 6e 66 09 32 09 31 09 31 09 67 70 61 09 2d |8.nf.2.1.1.gpa.-|
(base) Johns-iMac-Pro :: Data/Research/ignored_stuff »
You can DM me your file and I can have a look, if you want.
1 Like
Having some trouble figuring out how to DM you, but happy to send along an example file. I opened up the file in my text editor, Atom, and search for spaces, and found none. Also ran the hexdump command and got the following output below. I searched through the entire output. I didn’t find any 20s, and I did find many 09s.
MacBook-Pro-2:func flutist4129$ hexdump -C sub-MWMH378_ses-1_task-faces_events.tsv | head 00000000 22 6f 6e 73 65 74 22 09 22 64 75 72 61 74 69 6f |"onset"."duratio| 00000010 6e 22 09 22 74 72 69 61 6c 22 09 22 66 61 63 65 |n"."trial"."face| 00000020 22 09 22 62 6c 61 6e 6b 22 09 22 66 69 78 22 09 |"."blank"."fix".| 00000030 22 66 65 6d 61 6c 65 22 09 22 68 61 70 70 79 22 |"female"."happy"| 00000040 09 22 69 6e 74 65 6e 73 69 74 79 31 30 22 09 22 |."intensity10"."| 00000050 69 6e 74 65 6e 73 69 74 79 32 30 22 09 22 69 6e |intensity20"."in| 00000060 74 65 6e 73 69 74 79 33 30 22 09 22 69 6e 74 65 |tensity30"."inte| 00000070 6e 73 69 74 79 34 30 22 09 22 69 6e 74 65 6e 73 |nsity40"."intens| 00000080 69 74 79 35 30 22 09 22 63 6f 72 72 65 63 74 22 |ity50"."correct"| 00000090 09 22 70 72 65 73 73 22 0a 31 34 2e 35 09 32 2e |."press".14.5.2.|
I also opened the file in Excel, and it looks fine.
For comparison I ran hexdump on a random events.tsv from bidsexamples:
~/projects/bids/bids-examples$ hexdump -C ./ds011/sub-11/func/sub-11_task-Singletaskweatherprediction_run-01_events.tsv | head -n 1
00000000 6f 6e 73 65 74 09 64 75 72 61 74 69 6f 6e 09 74 |onset.duration.t|
Only difference I’m seeing is the lack of quotes, when I wrap that files onset header in quotes it throws error code 20.
I’ll make an issue for the quotes not showing in the validators evidence output.
1 Like
Ah! Beautiful. I added the quote=FALSE argument to my call of write.table(), and now I am no longer getting the errors.
1 Like
