Hi All, I recently ran a searchlight and am attempting to identify significant clusters that reliably predict above chance and above behavior (accuracy of ~63%). I’m a bit unsure how to check for significance in either situation. I’ve been using Nilearn and understand how to look for clusters abov…

Your procedure may provide meaningful results to your eyes, but you cannot claim significance. To claim significance, you need to run a formal permutation test on the whole brain or use some kind of Bonferroni correction. I know that this is not great, but this is a core issue with searchlight app…

Is there a reason I can’t do a permutation test only using the areas of interest?

If your ROIs were really selected independently from your current data, indeed you can restrict the test to these regions. Best, Bertrand

I meant after I run my searchlight only some voxels are predictive above some threshold. Can I only run my permutation tests on the predictive voxels rather than whole brain? I think I’ve seen this done in papers before and would make the computational cost tractable for me. Otherwise we’d be looki…

No, that’s double-dipping and you will get many false positives. I’d suggest having a look at: [image] Cluster correction for partial correlation on searchlight map Dear experts, I am not sure whether this is the right forum for this question, so if not I ho…

Ah so I’d end up assuming results are more significant than they truly are? Is the idea that I should be trying to see how often clusters of this size + this threshold occurs in every voxel not just the probability of this occurring in THESE direct voxels? Is that right?

Basically, yes. You want to know how unlikely your result is to occur if there is no signal to detect and the method is simply producing numbers out of its null distribution. The more complicated the statistics, generally, the harder it is to predict what that null distribution looks like; hence per…

Searchlight Cluster-Level Significance

effigies August 30, 2024, 6:44pm 6

No, that’s double-dipping and you will get many false positives. I’d suggest having a look at:

1 Like