RSA Searchlight analysis with high baseline values

Hi all, I’ve used rsatoolbox to compute a searchlight based on rsa between a model and some beta maps. I’d like to do a second level analysis on it, but I found that the baseline similarity is quite high with almost all voxels within the gray matter mask, which causes the significance results from permutation test output to be all significant.

I’ve never seen this discussed anywhere, presumably because usually similarity is not as high all over the brain. I’m trying to come up with a way to “clip” the similarity so that it’s not universally high. One way I can resolve this could be to simply remove the mean similarity across the brain and clip at 0. This way the baseline values would just be 0 and isolated clusters actually would mean something.

I understand this is a rare problem and my language might not be the clearest. Please let me know if you require more clarification. Thank you.