DICOM handling in DataLad

Software Support
,
1

What is the recommended approach for handling DICOM data in datalad as inputs for conversion to BIDS via heudiconv? My question is similar to this post: Datalad performance with dicom files

Do the dicom directories simply need to be compressed prior to saving them to the Datalad dataset?

I’m not finding guidance on this in https://github.com/datalad/datalad-neuroimaging . I found another resource that appears to be outdated: Example queries — Datalad for Hirnis 0.0.7 documentation

Please advise on the best approach to track DICOMS throughout the conversion workflow.

Thanks!
Meaghan

2

Hi @MeaghanPerdue . It is a great topic and we do not have a universal workflow yet … I would say, Michael Hanke likely to have some some more recent workflow formalized, and @bpinsard has efficient workflow AFAIK, and I am yet to get back to that part to implement locally here as well, but overall, it seems to scale up easily, while allowing for archival etc. Just briefly the best strategy is overall to

  • produce per MRI session a dataset with DICOMs

  • they could/should overall be first placed into a tarball one way or another (ria or just direct archival + datalad add-archive-content or alike), deposited somewhere, and thus indexed back within git-annex

  • those then should be installed and get’ed as needed within target dataset under sourcedata/ somewhere (may be sourcedata/dicoms/) to convert from

  • heudiconv could still add extra archives via its heuristics operation to simplify then fining associated dicoms

3

Thanks @yarikoptic ! We have a workflow where dicoms from the whole session are grouped in a 7z file (with 1 file per lzma2 block, allowing in theory random access to individual dicom, though this is not what datalad-archive does by default. Anyway most of the time we want all dicoms from 1 session to do the conversion).

The 7z file is added to the dataset and then indexed with datalad-add-archive, except for some GE scanners that do not have either enhanced or mosaic dicoms and result in 200k+ dicoms files (1 for each slice, even for dwi of fmri ) per session that ultimately completely breaks git and datalad. In that case we have a dataset that only tracks a single archive and a few config files.

Only the 7z file is stored in a datalad remote-storage, so to get a single dicom the full archive needs to be retrieved. Maybe a intermediate strategy could be to create an archive per series.

With enhanced dicoms it would be feasible to store dicoms directly in datalad, just make sure that the gitattributes are set to annex these properly. The only downside would be not to benefit from compression. The redundant and very verbose headers are likely very compressible, but the voxel data somewhat less. As this data end-up to be mostly dormant after conversion, a high compression ratio, even at a higher compression/decompression cost (time/cpu/mem) is advisable to optimize storage costs.