We were recently trying to reproduce an analysis, beginning by re-preprocessing all of the subjects. We assumed that we’d be able to reproduce our results identically, but it seems to be the case that some of the confounds that fmriprep outputs are not identical. Of course, they are incredibly close and perhaps we are wrong to assume that we’d be able to get identical results when reproducing our analyses. In any case, I think we have pinpointed the potential problem with the creation of the masks that are used to extract CSF, GM, and WM. These masks are slightly different each time fmriprep runs, which appears to lead to the non-reproducible results. Is this expected behavior? Or are we missing something?
Please let me know if I can provide further information. We’ve confirmed that fmriprep v20.2.3 was used for each analysis, and the scratch directory and fmriprep output directory were both cleared after each iteration of fmriprep.
Yes, this is consistent with previous reports by Prof. Milham’s team. Indeed, those masks might vary in some on/off voxels around the edges, which then introduces some run-to-run variability into some of the confound timeseries.
One question is whether you set the exact same seeds for the re-test run. fMRIPrep stores the seed it used in the config file under the work directory. If you reuse that seed, you should not get different results.
We would also greatly appreciate if you give us more information about your assessment - you seem to have done a rigorous investigation on the masks, and that might help us find additional sources of variability we had not identified yet.
For a start, can you please report the command-line you executed (mostly, I want to learn whether you are using --fs-no-reconall or not.