I am conducting actually a multi-voxel pattern analysis on several ROIs (x6), and several conditions (2 groups, 2 classes). I obtained accuracies (following a cross-validation scheme) and I need to know which ones are above chance level.
I would like to use permutations at the group level instead of binomial tests or t-tests. Then I conducted 1000 permutations for each condition and subjects.
My questions are:
- What is the best way to compute my null distribution (e.g., randomly pick permutation values within each of my subjects)?
- Do I have to compare the mean accuracy or the distribution of accuracy against the null distribution?
- Do I have to adjust then each p-value for multiple comparisons (e.g., FDR)?
I hope my questions are clear…