Error using bids_validator

bids

#1

Hi experts,
I am having this error when trying to run bids_validator on my folder containing MEG data in BIDS format:
(running under MacOS 10.12.6, and I converted my MEG data using mne-bids (python 3.6).
Any ideas?
Thanks a lot!

(mne) MEGdata $bids-validator WM_MEG_BIDS_2

<--- Last few GCs --->

[34938:0x104000000]   459456 ms: Mark-sweep 1150.5 (1158.1) -> 1150.4 (1157.6) MB, 33.3 / 0.0 ms  (average mu = 0.999, current mu = 0.001) last resort GC in old space requested
[34938:0x104000000]   459467 ms: Mark-sweep 1150.4 (1157.6) -> 1150.4 (1157.1) MB, 10.2 / 0.0 ms  (average mu = 0.998, current mu = 0.001) last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x2159cf65be3d]
Security context: 0x11a380c1e6e1 <JSObject>
    1: stringSlice(aka stringSlice) [0x11a3a1950669] [buffer.js:594] [bytecode=0x11a3c3d24229 offset=91](this=0x11a35e8026f1 <undefined>,buf=0x11a356660b29 <Uint8Array map = 0x11a34eed1eb1>,encoding=0x11a380c3d819 <String[4]: utf8>,start=0,end=440563208)
    2: toString [0x11a3c591e059] [buffer.js:667] [bytecode=0x11a3c3d23d19 offset=145](this=0x11a356660b29...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: 0x10003b125 node::Abort() [/usr/local/bin/node]
 2: 0x10003b32f node::OnFatalError(char const*, char const*) [/usr/local/bin/node]
 3: 0x1001a8e85 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 4: 0x1005742a2 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/usr/local/bin/node]
 5: 0x10057d7a4 v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/usr/local/bin/node]
 6: 0x10054f594 v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/usr/local/bin/node]
 7: 0x10054f1d9 v8::internal::Factory::NewStringFromUtf8(v8::internal::Vector<char const>, v8::internal::PretenureFlag) [/usr/local/bin/node]
 8: 0x1001cb228 v8::String::NewFromUtf8(v8::Isolate*, char const*, v8::NewStringType, int) [/usr/local/bin/node]
 9: 0x1000ddc1a node::StringBytes::Encode(v8::Isolate*, char const*, unsigned long, node::encoding, v8::Localv8:Value>*) [/usr/local/bin/node]
10: 0x100053f9b void node::Buffer::(anonymous namespace)::StringSlice<(node::encoding)1>(v8::FunctionCallbackInfov8::Value> const&) [/usr/local/bin/node]
11: 0x10023170f v8::internal::FunctionCallbackArguments::Call(v8::internal::CallHandlerInfo*) [/usr/local/bin/node]
12: 0x100230c51 v8::internal::MaybeHandlev8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<false>(v8::internal::Isolate*, v8::internal::Handlev8::internal::HeapObject>, v8::internal::Handlev8::internal::HeapObject>, v8::internal::Handlev8::internal::FunctionTemplateInfo>, v8::internal::Handlev8::internal::Object>, v8::internal::BuiltinArguments) [/usr/local/bin/node]
13: 0x1002302f0 v8::internal::Builtin_Impl_HandleApiCall(v8::internal::BuiltinArguments, v8::internal::Isolate*) [/usr/local/bin/node]
14: 0x2159cf65be3d 
Abort trap: 6

#2

This is a known error that we are currently trying to solve: https://github.com/bids-standard/bids-validator/issues/677. Unfortunately we are struggling to reproduce it. Could you by any chance share the dataset (or a subset of it that still runs into the error)? dropbox or google drive would do


#3

Yes. Here is the link to a subset of the folder that still runs into the error:
https://drive.google.com/open?id=1wDi8ArfkCEvZP40k7o4FEdIsfH0LhIMX


#4

Hi @romain_quentin

This may help. There seems to be a space between “Working Memory” in your names. That could be causing a hiccup with BIDS validation. Please see below.

Capture

Also, it may be worth relocating your files to a new directory to see if the memory issue resolves. This worked for my fMRI data. I am not sure why it worked, but I’m currently working on finding out why the issue was resolved by changing directories.

Good luck!


#5

Hi @TribikramT,
Thank you for your feedback!
Unfortunately, removing the space in the task name and relocating my files to a new directory gives me the same error message…


#6

Could you try https://2001-37161308-gh.circle-artifacts.com/0/root/web_version/index.html or

npm install -g git+https://git@github.com/DaNish808/bids-validator.git#more-memory-#677


#7

Great. I have others errors but different (and seems to be related to my files).
I will try to arrange my files to pass the test and maybe come back to you :slight_smile:
Thanks a lot!

	1: [ERR] Empty files not allowed. (code: 99 - EMPTY_FILE)
	./sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_run-01_meg.ds/BadChannels
	./sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_run-01_meg.ds/YFAALKWR_wmtask_20160226_01.eeg
	./sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_run-01_meg.ds/bad.segments
	./sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_run-01_meg.ds/hz.ds/BadChannels
	./sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_run-01_meg.ds/hz2.ds/BadChannels
	./sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_run-02_meg.ds/BadChannels
	./sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_run-02_meg.ds/YFAALKWR_wmtask_20160226_02.eeg
	./sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_run-02_meg.ds/bad.segments
	./sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_run-02_meg.ds/hz.ds/BadChannels
	./sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_run-02_meg.ds/hz2.ds/BadChannels
	... and 31 more files having this issue (Use --verbose to see them all).

1: [WARN] Tabular file contains custom columns not described in a data dictionary (code: 82 - CUSTOM_COLUMN_WITHOUT_DESCRIPTION)
	./sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_run-01_events.tsv
		Evidence: Columns: event_value, event_sample not defined, please define in: /sub-01/ses-01/meg/sub-01_ses-01_events.json, /sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_events.json,/sub-01/ses-01/sub-01_ses-01_events.json,/sub-01/ses-01/sub-01_ses-01_task-WorkingMemory_events.json,/sub-01/sub-01_events.json,/sub-01/sub-01_task-WorkingMemory_events.json,/events.json,/task-WorkingMemory_events.json,/sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_run-01_events.json
	./sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_run-02_events.tsv
		Evidence: Columns: event_value, event_sample not defined, please define in: /sub-01/ses-01/meg/sub-01_ses-01_events.json, /sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_events.json,/sub-01/ses-01/sub-01_ses-01_events.json,/sub-01/ses-01/sub-01_ses-01_task-WorkingMemory_events.json,/sub-01/sub-01_events.json,/sub-01/sub-01_task-WorkingMemory_events.json,/events.json,/task-WorkingMemory_events.json,/sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_run-02_events.json
	./sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_run-03_events.tsv
		Evidence: Columns: event_value, event_sample not defined, please define in: /sub-01/ses-01/meg/sub-01_ses-01_events.json, /sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_events.json,/sub-01/ses-01/sub-01_ses-01_events.json,/sub-01/ses-01/sub-01_ses-01_task-WorkingMemory_events.json,/sub-01/sub-01_events.json,/sub-01/sub-01_task-WorkingMemory_events.json,/events.json,/task-WorkingMemory_events.json,/sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_run-03_events.json
	./sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_run-04_events.tsv
		Evidence: Columns: event_value, event_sample not defined, please define in: /sub-01/ses-01/meg/sub-01_ses-01_events.json, /sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_events.json,/sub-01/ses-01/sub-01_ses-01_events.json,/sub-01/ses-01/sub-01_ses-01_task-WorkingMemory_events.json,/sub-01/sub-01_events.json,/sub-01/sub-01_task-WorkingMemory_events.json,/events.json,/task-WorkingMemory_events.json,/sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_run-04_events.json
	./sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_run-05_events.tsv
		Evidence: Columns: event_value, event_sample not defined, please define in: /sub-01/ses-01/meg/sub-01_ses-01_events.json, /sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_events.json,/sub-01/ses-01/sub-01_ses-01_events.json,/sub-01/ses-01/sub-01_ses-01_task-WorkingMemory_events.json,/sub-01/sub-01_events.json,/sub-01/sub-01_task-WorkingMemory_events.json,/events.json,/task-WorkingMemory_events.json,/sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_run-05_events.json
	./sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_run-06_events.tsv
		Evidence: Columns: event_value, event_sample not defined, please define in: /sub-01/ses-01/meg/sub-01_ses-01_events.json, /sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_events.json,/sub-01/ses-01/sub-01_ses-01_events.json,/sub-01/ses-01/sub-01_ses-01_task-WorkingMemory_events.json,/sub-01/sub-01_events.json,/sub-01/sub-01_task-WorkingMemory_events.json,/events.json,/task-WorkingMemory_events.json,/sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_run-06_events.json
	./sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_run-07_events.tsv
		Evidence: Columns: event_value, event_sample not defined, please define in: /sub-01/ses-01/meg/sub-01_ses-01_events.json, /sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_events.json,/sub-01/ses-01/sub-01_ses-01_events.json,/sub-01/ses-01/sub-01_ses-01_task-WorkingMemory_events.json,/sub-01/sub-01_events.json,/sub-01/sub-01_task-WorkingMemory_events.json,/events.json,/task-WorkingMemory_events.json,/sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_run-07_events.json
	./sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_run-08_events.tsv
		Evidence: Columns: event_value, event_sample not defined, please define in: /sub-01/ses-01/meg/sub-01_ses-01_events.json, /sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_events.json,/sub-01/ses-01/sub-01_ses-01_events.json,/sub-01/ses-01/sub-01_ses-01_task-WorkingMemory_events.json,/sub-01/sub-01_events.json,/sub-01/sub-01_task-WorkingMemory_events.json,/events.json,/task-WorkingMemory_events.json,/sub-01/ses-01/meg/sub-01_ses-01_task-WorkingMemory_run-08_events.json

    Summary:                 Available Tasks:        Available Modalities: 
    285 Files, 3.34GB        WorkingMemory           01                    
    1 - Subject                                      02                    
    1 - Session                                      03                    
                                                     04                    
                                                     05                    
                                                     06                    
                                                     07                    
                                                     08

#8

Are the empty files recognized correctly or are those false positives?


#9

They are recognized correctly.
MEG CTF data acquisition system creates automatically these files (but if they are not bad channels, or bad segment of data, these files are empty).
Is there any way to accept empty files? (I can delete them but some participants (run or session) may have bad channels and then will need these files).


#10

#11

Are the empty files recognized correctly or are those false positives?

see also https://github.com/bids-standard/bids-validator/issues/651