Permutation in Searchlight analysis [Nilearn]

, , ,
1

Hello,

I am currently performing a searchlight based classification using nilearn.decoding.Searchlight decoder and everything works fine, but I would like to be able to produce some chance maps (for my statistics as in e.g. Stelzer et al., 2013) by permuting the labels of my training sets and keeping the test set unchanged. Is there an easy way to achieve that? Up to now I am only able to shuffle both training and the test set but that’s not what I want.

Thanks!!

2

Hm if I understand correctly the DummyClassifier would permute the labels for you for the training set and in order to be able to “fix” the test set you would need to adjust the “group” parameter of the cross validation as that the Test set is that same in all the folds?

4

Thank you for your reply! I am not sure I quite get this approach. How would I implement this after doing the searchlight via nilearn?

5

Note that the dummy classifier indeed returns chance-level accuracy, but id does not rely on the permutation scheme you describe. I don’t think that Nilearn API can be used to do exactly what you describe.
On the other hand, I think that it is OK to shuffle labels on the test set.
HTH,
Bertrand