My neural data are recorded with Blackrock (.ns6, .nev) and my behavioural data are recorded with LabStreamingLayer->eXtensible Data Format (.xdf). I’m fairly confident my folders and files are named properly, yet BIDSLayout doesn’t find any entities. (I suspect this will change when I add my imaging data tomorrow, but that’s beside the point of the question).
I have no intention to convert my neural and behavioural data. I’m quite comfortable with my formats, they have open specifications, they have open source tools, and there’s no clear path to go from .ns6/.nev to any of the official formats without losing most of my data. The only possible format is NWB. python-neo loads Blackrock files very well (maybe their best-supported format) but doesn’t have nwb support yet (issue has been open for 5 years). There is a Matlab path to convert the data, but I don’t have Matlab (and this shouldn’t be a requirement).
I think I can define how to search for my files by providing a my_config.json file. I see the default files in bids/layout/config/ . Is there a way to simply extend the default path patterns? Or do I copy-paste all the contents of bids.json then add my own specifications? Is there a guide on how to write this file?
Thanks in advance,
Chad
Hi @cboulay, I would suggest first setting validate=False for your BIDSLayout and see how far that gets you. PyBIDS is not picky about extensions, and will happily index anything you like, but if validate=True, it will pass each path to bids_validator.BIDSValidator(), which does check extensions.
The reason to add a configuration JSON is if you have non-standardized entities (the filename is made up of (<entity>-<value>_)+<suffix>.<extension>) that you want to select over.
Re entities, I guess that also includes datatype? The data I’m working with are closest to ieeg which is not in the default bids.json even though it’s an accepted BIDS extension as far as I can tell.
(While drilling down to find the problem, I also discovered that it’s not matching my top-level file because ‘\\dataset_description.json’ is not in the list which includes ‘/dataset_description.json’. Does this whole thing not work in Windows?)
Yes, data type is an entity in the pybids sense, but not really by the BIDS standard. But the consequence of failing to match the datatype should simply be an inability to search/filter by datatype. The files should nonetheless match.
But if we’re not supporting valid BIDS directories, then we should. Would you care to open an issue or PR to add the ieeg data type?
We haven’t had anybody with a Windows machine to help get the tests running in Windows, so Windows support is at best accidental, but probably just absent.
OK I will create an issue for ieeg and a PR for a simple fix to at least get some of the Windows stuff working (os.pathsep --> ‘/’). I don’t have the confidence to make a PR for ieeg because I still don’t quite understand how the bids.json layout config file is intended to work. Maybe a request for documentation there is another issue.
I’ll mark your first answer as a solution once I can get all my files in a BIDSLayout object.
1 Like