I would like to conduct a group level analysis on a searchlight decoding analysis. I have fMRI data of participants who listened to two distinct syllables and I would like to infer the brain areas that can distinguish between syllable A and syllable B.
Following the TDT demos, I first ran the searchlight decoding analysis for every participant with a cross-validation design. For this I used normalized, but unsmoothed data. Afterwards, I ran permutations on every participant. The permutations I ran in the combined mode to save time. Like this I got a res_accuracy_minus_chance_set00XY.mat for every permutation.
I have a two questions before starting with pervalence statistics:
Would you smooth the decoding data and the permutation data of the participants beforhand? This is a naïve question, and it may not apply to the prevalence statistics. I am just asking because I read that you gave this suggestion for classical second level statistics (t-tests etc.), but I did not find any reference to this for the prevalence inference in the TDT demo scripts.
I realized that even though, I used normalized data and a group mask.nii file for the searchlight decoding the res_accuracy_minus_chance.mat vectors have not the exactly the same length across participants. When trying to run pervalenceTDT, I realized that this is problematic
First, why do my res_accuracy_minus_chance.mat vectors have different lengths? And second, would you recommend to use the .nii files instead? If I would decide to use the .nii files, I would need to tranfer res_accuracy_minus_chance_set00XY.mat from every permutation into a nii image, right? Is there a easy way to do this?
Yes, I recommend smoothing because this will reduce the discreteness associated with accuracies, but is still valid insofar as you believe information to be not only tied to individual voxels but to regions (which you assume with searchlights anyway). This is helpful for standard t-tests to get more normally-distributed accuracies, but it can also help in case you have only a limited number of discrete accuracy bins (e.g. for 10 samples 0 0.1 0.2 … 1) which can lead to overly conservative permutation statistics without smoothing.
This might be an issue with SPM. I would check that the group mask contains only voxels that are present in all participants.
Oh, they weren’t written as a default? Hmm. Let me know if you go with the .mat files. I’m a little busy right now (sorry!), but if you need the .nii files, I’m more than happy to guide you there!
I will checked that mask again, I think the problem could be that I was derived from an analysis where I used smoothed single subject data for group level stats.
Concerning, the prevalence statistics, I just discussed the pipeline with a colleauge and he was a bit puzzled by the fact that I would be needd to smooth the permutation data of every participant for every permutation, but maybe that was not what you meant. I am sorry that I have to ask for clarification concerning the pipeline again.
Was this the pipeline you recommended?
(1) Decoding (searchlight)
on normalized unsmooted data
(2) Permutation (searchlight)
on the same normalized unsmooted data
(4) smoothing of the decoding data per participant / and for every participant for every single permutation?
(5) Prevalence group level analysis
on smoothed decoding data and smoothed permutation data
Yes, this is the pipeline (with one exception, see below). Regarding step (4) I generally recommend smoothing searchlight accuracy maps for statistical reasons (improves local estimates of accuracy, reduces effect of inaccuracies in spatial normalization between participants, and for random field theory approaches makes results more consistent with the assumptions). If you smooth your main results, this means you would have to smooth permutation-based accuracy maps, as well.
You could also contact Carsten Allefeld, who wrote the original paper.
I generally recommend first running a searchlight analysis and then normalizing and smoothing results. The main reason is that it’s typically a lot faster (because you have fewer voxels to consider before normalization). Other than that the order shouldn’t affect results much, so if you already have your volumes, then you are good.
One thing to bear in mind with prevalence inference that I’m currently pondering on: It’s a valid inference approach for group-level analyses in MPVA, but it means your conclusions rest on the assumption that you have “good” subjects. If you have bad subjects, they may not carry an effect, not because it is not there but because they are too noisy. That, in turn, will affect your estimate of the percentage of participants carrying an effect. I think this is worth considering, because it affects your conclusions. In other words, it’s worth discussing whether setting gamma = 0.5 (i.e. at least 50% of the participants carry an effect) is the only way to do prevalence inference. Ultimately, the prevalence criterion is a subjective choice that you would need to justify. Happy to hear other thoughts on this, though.