I am pretty new to nilearn and to working with fMRI data in general, so please excuse if this question is too simple.
I have run a searchlight analysis for each of my participants individually and now I would like to somehow combine these individual analysis to a group analysis. I would then like to use this group result as a feature extraction method. So I would like to afterward train models only on the features (voxels) which have high accuracy for all participants. My questions are:
Does this procedure make sense in general?
How can I combine the in individual searchlight results in nilearn?
thank you very much for your question and welcome to Neurostars, it’s great to have you here.
Both aspects you mentioned are highly debated since the “early days” of machine learning analyses applied to neuroimaging data. Going question by question here:
This depends: are you using the extracted features on new/unseen data? For example, using the regions with high accuracy to create ROIs that are then used to run/train/evaluate models on data that was not used to create the ROIs? If not, ie. you’re using the same data for feature extraction and then running/training models then this might be considered “double dipping” and won’t be “statistically sound”. First, the data wouldn’t be independent and second, you already know that this voxels/regions contain “information” necessary/sufficient to distinct whatever you’re trying to differentiate. Thus, it would be a bit biased in terms of increasing the outcome measure (e.g. accuracy). However, all of this is based on assumptions and I might have misunderstood things. BTW: did you evaluate other options of feature selection, for example apriori ROIs (from meta-anlyses, based on structure, etc.) or an F-test (that’s implemented in scikit-learn and can thus comparably easily be used in nilearn).
As with basically all other aspects of neuroimaging analyses, there are quite a few proposals/suggestions to do this. Some folks use the individual searchlight maps in classic inference statistical test, e.g. t-test or f-test, other use the Stelzer et al. approach: combing permutation tests and cluster thresholds based on bootstraps (not available in nilearn but in pymvpa), run cross-validation and permutation tests across participants, etc. . Would it be possible to provide a bit more information on your planned analyses/the goal thereof?