Apologies if this is a repeat topic, but I couldn’t find anything in a search. Is there a protocol for removing unneeded scans (e.g., aborted functional runs, bad T1’s, etc.) from a bids directory?
We are set up to have all DICOM series transferred from a central server to our analysis server and automatically converted into bids format. This means that all data are converted into bids format, even for runs that we do not want. We are specifically running into problems where a subject has had multiple re-takes of a T1-scan. We would like to remove the bad runs so that fmriprep only uses the good run during the preprocessing of the functional data, however, just removing the bad runs from the bids folder (not deleting, just relocating to a directory outside the bids directory) still results in fmriprep trying to process these bad runs.
For the unneeded scans one way could be to look at the metadata (i.e. pixel dim) because it will likely differ between your good and bad scans. You may consider looking at that and I am assuming your good data will all have the same (or nearly the same) metadata headers. Before running fMRIPrep you could also run MRIQC to go through and see the scan quality. This would happen between the BIDS conversion and running fMRIPrep. It is interesting that removing the scans from the directory before running fMRIPrep, that fMRIPrep has memory to run the bad runs.
Thanks for the response. The pixel dim works well enough for the functional runs. Although this particular study, two of the tasks are self-paced, so there is no single number of time points that can be easily used as a criterion in the heuristic file to separated aborted runs from completed runs.
The bigger problem we are having is with a subject having to go through multiple T1 scans, as the techs typically do not abort these scans prematurely (don’t know until the data are reconstructed on the scanner if the T1 is useable or not). So, there is no difference in the meta data for bad T1 scans versus good T1 scans, making it difficult to write a heuristic file that will only convert the good T1 scans into BIDS format. Also, since we are set up to do an auto-pull of the DICOM data from our DICOM server, there is no easy way to indicate that only certain DICOM series should be pulled.
This means that we end up with multiple T1 scans that have been converted into BIDS format in the subject’s anat folder. When running fMRIprep, there is no way to tell fMRIprep to ignore the bad runs. And as I said in my first post, deleting them from the subject’s anat folder doesn’t seem to help. We have also tried deleting the file names from the subject’s sub-_ses-_scans.tsv file, but all that does is cause fMRIprep to create a new anat folder with all of the ‘fmriprepped’ T1 files. The ‘anat’ folder located within the session folder (at the correct level with the ‘func’ folder) just has the affine transform .txt file.
We think there must be another record of these bad T1 scans in other .tsv file, but we cannot figure out where in all of the heudiconv output…
It appears the clean up will happen with the niftis (because of the auto-pull/conversion to BIDS there is no time to intervene on the DICOMs before the conversion). The clean up can happen before fMRIPrep is run. I am not sure what .tsv information fMRIPrep may use (@effigies@oesteban?)
You may consider using MRIQC. This would be able to help identify those bad scans (and general scan quality) so it can be cleaned up before fMRIPrep is ran. So the result will be just the good scans that will be given to fMRIPrep to run. It is interesting that fMRIPrep will run on the bad scans even though there is nothing there and has not seen those bad scans before? (Even with a subject’s clean directory of just the good scans?) One thing I can see is having to also correct the filename after cleaning up and before fMRIPrep (the multiple scans may have heudiconv add a run field to the name that may not be needed)
I’m a little confused about the situation, here. Removing files from the BIDS root directory should certainly prevent fMRIPrep from finding them. Where are you putting them? Could you share more details about your directory structure and fMRIPrep command?
With regard to .tsv files, fMRIPrep does not make use of any information in .tsv files. We do produce regressor.tsv files, but that’s it.