Bids validation Issue: JS stack trace

Hello,
I’ve been using fmriprep version 1.3.0 and recently came across this issue while processing some data:

<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x2a1c5b0dbe1d]
Security context: 0x0fd5b789e6e1 <JSObject>
    1: /* anonymous */ [0x3749ef752571] [/usr/lib/node_modules/bids-validator/utils/files/collectDirectorySize.js:~6] [pc=0x2a1c5b475dc3](this=0x18386c29ad11 <JSGlobal Object>,key=0x35b07ff69dc1 <String[6]: 721122>)
    2: arguments adaptor frame: 3->1
    3: forEach [0xfd5b7886de9](this=0x3749ef7525e9 <JSArray[863120]>,0x3749ef752571 <JSFunction keys.forEac...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x8dc1c0 node::Abort() [bids-validator]
 2: 0x8dc20c  [bids-validator]
 3: 0xad60ae v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [bids-validator]
 4: 0xad62e4 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [bids-validator]
 5: 0xec3972  [bids-validator]
 6: 0xec3a78 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [bids-validator]
 7: 0xecfb52 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [bids-validator]
 8: 0xed0484 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [bids-validator]
 9: 0xed30f1 v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [bids-validator]
10: 0xe9c574 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [bids-validator]
11: 0x113beae v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [bids-validator]
12: 0x2a1c5b0dbe1d 
Making sure the input data is BIDS compliant (warnings can be ignored in most cases).

Traceback (most recent call last):
  File "/usr/local/miniconda/bin/fmriprep", line 11, in <module>
    sys.exit(main())
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/cli/run.py", line 354, in main
    validate_input_dir(exec_env, opts.bids_dir, opts.participant_label)
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/cli/run.py", line 535, in validate_input_dir
    subprocess.check_call(['bids-validator', bids_dir, '-c', temp.name])
  File "/usr/local/miniconda/lib/python3.7/subprocess.py", line 341, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['bids-validator', '/N/dc2/scratch/dlevitas/fMRI_data/std/bids', '-c', '/tmp/tmpe3hy1dts']' died with <Signals.SIGABRT: 6>.
Sentry is attempting to send 1 pending error messages
Waiting up to 2.0 seconds
Press Ctrl-C to quit

The data passed the BIDS validator with no errors (just a few warnings), and the job I submitted for this data was 55gb, so I’m unsure why I’m getting these memory issues. To resolve the issue I’ve been adding --skip_bids_validation; however, I like to have the BIDS validation performed on each dataset if possible. I should note that I ran the data without validation, and the report did not generate any errors or issues. I was wondering if there is a way to fix this without having to turn off the BIDS validation step?

Thanks,

Dan

Hi Dan,

Could you open an issue over at the validator project? It would be good to specify how you were running fMRIPrep (e.g. a Singularity image vs a native installation), which will help identify the software version. Some statistics about your dataset might be useful, like the number of files and total dataset size. I don’t know Javascript errors well, but a heap-out-of-memory issue seems plausibly related to a very large dataset.

Best,
Chris

This was indeed a memory issue, given that this particular participant’s dataset was 5.4gb. Other datasets of similar size don’t have this problem though, so it’s difficult to determine the exact cause. My solution has been to use --skip_bids_validation in the rare circumstances when this issue occurs, but otherwise, perform the bids validation.

Dan

For posterity: