Most data standards assume you are starting a project. This is for the other
case: a drive arrives, the person who filled it has graduated, and three years
of recordings are in there with nobody left who can say which animal is which.
NDOS reads that drive without changing it, works out what is in it, and lets
you ask questions of the result. It is Python 3.9+ with no third-party
runtime dependencies, because the machine that most needs this is often an
acquisition PC where nothing can be installed.
pip install ndos
ndos report /path/to/the/drive
Where it sits relative to BIDS and NWB — the first question anyone asks
It does not compete with either, and it does not claim conformance it cannot
demonstrate.
ndos convert bids produces a BIDS-shaped export, and says so in exactly
those words:
BIDS-shaped, not validated BIDS. Animal electrophysiology is covered by
BEP032, which is not finalised, so no export can claim conformance today.
It maps to ephys, beh and micr. ndos convert nwb emits a conversion
plan with metadata mapped onto NWB’s fields, for NeuroConv or a lab script
to execute — NDOS does not reimplement NWB writing, which is solved and
maintained elsewhere.
There is a directory specification in the repository, but it is one component
and deliberately separable. If your lab already uses
NeuroBlueprint or has its own
convention, the inventory, search, metadata and provenance parts still work on
what you have. The specification names three levels for this reason:
readable (works on whatever organisation exists), conformant, and
traceable.
What 0.2.0 adds
Search that leads somewhere. You remember a word — CA1, a construct
name — and the thing that knows where it applies is a surgery log nobody has
opened in three years.
ndos search index /path/to/drive
ndos search find "CA1 AND injection"
It reads filenames, notes, protocols and spreadsheets, including .docx and
.xlsx (ZIPs of XML, so no parser dependency; PDFs are not read and it says
so). A hit in that surgery log reports which animals the text names, and
where those animals’ recordings are — so a word leads to the data, not just
to a document.
Checking a transfer arrived intact. NDOS does not move data; rsync,
rclone, DataLad and datashuttle do that well. What none of them reports is
whether what arrived is still readable.
ndos scan /source -o before.json
# ... transfer however you already do ...
ndos verify before.json /destination
Three results rather than two — VERIFIED, FAILED, and INCONCLUSIVE for
when no mismatch was found and nothing was proven either (a manifest with no
checksums, a path neither end could read). A Kilosort sorting that arrived one
file short is reported as an unopenable sorting rather than as one missing
path, because that is what it is. A .bin that arrived without its .meta is
flagged for the same reason.
The part I would most like argued with
Cohort queries return three answers, not two: matched, excluded, and cannot
be ruled out.
A session whose species was never recorded is not a session known not to be a
mouse. Collapsing those two cases gives you a cohort quietly biased toward
whichever animals happened to have fuller records, and nothing in the output
says so. A worked example is in the repository
(examples/cohort-demo),
with committed output: four animals, one query, one match, one excluded, and
two undecidable for two different reasons — one checked and unknown, one
never entered.
I would genuinely like to hear from anyone who thinks this is the wrong model,
or who has a case it handles badly.
Status, plainly
489 tests on Python 3.9 and 3.13 across Linux, macOS and Windows, including a
CI job that runs every module against a bare interpreter so the
zero-dependency property cannot lapse quietly.
No lab outside my own has completed a pilot. That is what I am looking
for. It takes about fifteen minutes and will not move or change your data, and
the single most useful thing anyone could send me is a folder layout it
reads wrongly — the inference rules have met too few real conventions to be
trusted yet.