Cross-validation scheme for MVPA on a single run

, , , ,
1

Hello everyone,

I am trying to do a searchlight analysis on a dataset, in which there is one run per participant. I have a block design with two conditions (6 blocks each, approx. 10s between blocks). All the toolboxes I checked out seem to default to leave-one-run-out CV, which makes sense. However, I am trying to figure out how to do the searchlight when I only have one run per participant available. I was thinking of doing a leave-one-trial-out CV scheme but cannot figure out how to implement it.
I’ve been through the documentation of PyMVPA, CosmoMVPA and the decoding toolbox, but have been so far unsuccessful. If anyone has advice on how I may go about this I’d be very grateful!
Alternatively I was wondering if it would be useful looking into a leave-one-subject-out CV scheme. Again any advice is appreciated :slight_smile:

2

Check out nilearn’s searchlight. I think with that you can use any sklearn-compatible CV scheme. For example you can use the LeaveOneGroupOut CV generator. Then you should be able to define the groups as trials in the fit method or some such thing.

3

Great, thanks for the tip!