Replicable scripts, BIDS, and curating data



I’m currently analyzing a dataset, and I would like for each step of the analysis to be completely automated, so that I could publish this dataset and have the analysis be replicated exactly. (I’m not editing freesurfer surfaces, so I shouldn’t actually need to do a lot of interacting with the pipeline.)

The BIDS framework and all of its apps make that pretty simple – I run heudiconv on a list of subjects to get BIDS directories. I run MRIQC and fmriprep to check data quality and do preprocessing. Then I have a nipype modeling script, et cetera.

The thing I’m struggling with is this: in most large projects with a bunch of subjects, you’ve got some one-off subjects that you need to exclude. Maybe there’s an excessive amount of motion and the data is garbage. Maybe there’s a run where the projector turned off midway through. Maybe there’s one subject with a really unfortunately slice prescription that you don’t want to include in group analysis because the intersection of his mask with the other masks excludes too much data.

How is this documented and managed? Is there a BIDS standard for this? I’d like to keep the data in the dataset that we upload, and even if I didn’t, I wouldn’t want to deal with this in the heudiconv heuristics file (“if TRs = 128 and task = ‘faces’, process it, unless it’s the 2nd run of subject 8, or the 3rd run of subject 10, or …”)

Ideally, there’d be something like an “excluded runs” file somewhere so that there was documentation of the bad runs in a standardized place, and also so that by the time modeling scripts were active, they could intelligently exclude running first-level models on garbage data.

Has anyone done this in a clever way?


The core of this problem is that “exclusion of runs” is your particular interpretation of quality of the data which is dependent on what tools you used to asses it and what you are planning to use the data for. So the answer which runs to keep will differ from one person to another and from one analysis to another (T1w scans with some motion could be good as intermediate coregistration target, but not good for cortical thickness measurements).

At the moment the spec does not specify how to do this, but you can do the following:

  1. Add a known issues section to the README describing what you found problematic about specific runs
  2. Add a custom column to_scans.tsv files denoting which scans should be excluded or not. Add a_scans.json data dictionary explaining what this column means and how you made the decision.

This might be also a good thing to add to the spec. If you could propose a change on that would be great.


Is this something that has been implemented/could be accessed when running fMRIPREP? For example, I have some subjects who have two T1’s, and one of them has much more motion distortion than another, so I’d like to have FMRIPREP only use the good T1 rather than averaging them together. How could I point fMRIPREP to that column in order to decide which T1 to use?

I know I could just put a number of specific subject file names into the .bidsignore file, but that doesn’t seem to be the best long-term solution. We have a similar situation with some rest scans, where we want to exclude one of two rest scans when we saw that the subject fell asleep.


This would be an interesting new feature. However, FMRIPREP currently does a robust averaging of T1w images that excludes outlier voxels. Check if the output volume looks good. Perhaps manual exclusion is not necessary.

I agree (and I’m not sure if FMRIPREP uses .bidsignore anyway). A solution based on a specific column in _scans.tsv would probably be best.


Thanks, @ChrisGorgolewski! I’ll check how it runs on a few people with that situation.

As for the rest scans (wanting to ignore specific scans where subjects fell asleep), I’ll see if the .bidsignore workaround does anything at all. I think it is looked at at some point, as I used it to ignore fieldmap scans that I don’t want to use (before I realized there was a flag for that). But the column idea seems a much better solution longterm.