I am quite new to MVPA and have a question about design/cross-validation in TDT.
My study is separated into 6 runs with 2 runs per condition. There is only one condition per run, as it is a block design.
Using the leave-one-run out cross-validation seems to cause imbalanced test data.
As far as I understood, my classifier will be trained on all conditions but only tested on one. I feel that this is probably not the best way to do the classification.
From the posts I have read on this website, I conclude that using xclass could be a solution to my problem. This way, my classifier would be applied to all test data.
I just want to check whether this would be the appropriate approach? Any advice is much appreciated.
If I understand correctly, you have 3 conditions, where there are 2 runs with each condition and a total of 6 runs, and your aim is to see where there is information about these three conditions? That’s a tricky scenario. I would suggest a decoding design where you leave out half, i.e. assuming this is your setup
1 2 3 1 2 3
and run numbers
1 2 3 4 5 6
then I’d suggest to use for training
1 2 3
1 2 6
1 5 3
1 5 6
etc.
i.e. all valid permutations.
Since you would only have one data point for training, the classifier will end up being some sort of nearest neighbor approach. You could in principle also just use correlation_classifier.