MVPA searchlight based decoding

Hi Martin and all,

I am trying to perform a searchlight-based 8-way classification. I have 8 conditions and overall 512 beta estimates (64 betas for each condition). The estimated time to perform MVPA in the gray matter (~28,000 searchlights) is approximately four months.

RSA performed on the same dataset takes 40-60s depending on the number of searchlights.

Do MVPAs (8-way classification with 512 betas) usually take that long?

Vinodh

I think this definitely sounds unusually long. How large is your searchlight? If it’s really really small it might take a lot longer. Internally, this is running 28 pairwise classification analyses, but it still shouldn’t take much longer than a few minutes.

I think it would help if you could provide more information about the settings you used for the analysis (regarding the searchlight, the classifier, and the expected output). What you could also do is run “profile on” in the command window, then start the analysis, after 5min cancel the analysis, then run “profile off” and finally “profile viewer”. That should tell you where the bottleneck is. Feel free to report back!

One thing that helps if you are dealing with unusually large betas or betas with very different scale is to scale the data. Check out the scaling options in decoding_scale_data. This can often speed things up a lot.

Hope this helps!
Martin

Thank you Martin for the prompt reply.

I was using the following settings for my analyses-
Searchlight radius: 3 voxels
Classifier: 8-way classifier (to classify 8 graphemes)
Expected output: accuracy minus chance

Profile viewer showed ‘libsvm_train.m’ and ‘libsvm_test.m’ to be taking up the most time.

The analyses reduced to an 40-90 mins once I increased the searchlight radius to 15 voxels (while keeping all the other settings the same). Nevertheless, we do no intend to use pursue with that large radius.

Is there a principled way to choose the searchlight radius given the number of classifications?

Vinodh

Hi Vinodh,

Apologies for the delay, I was on a vacation and only just returned. Regarding the searchlight radius: There is no principled approach for choosing a radius, it just depends on the size you would find appropriate. 12mm seems quite common (which is 4 voxels for 3x3x3 and 6 voxels for 2x2x2). Please make sure you really choose voxels and not mm!

Glad you found the bottleneck. It seems you a running the analysis on single trials. I would try scaling the data first to see if that improves the speed, using decoding_scale_data. I would also consider not using single trial estimates (64 betas per condition) but really only one beta per condition per run. If you really want to use single trial estimates and if scaling doesn’t help, then I’d encourage you to reduce the classifier cost c to 0.01 or 0.001. This can be done by setting the following:

for classification_kernel (which is an internal trick for speeding up the computation by precomputing the linear kernel)
cfg.decoding.train.classification_kernel.model_parameters = '-s 0 -t 4 -c 0.01 -b 0 -q';
and if you don’t want to use that trick:
cfg.decoding.train.classification.model_parameters = '-s 0 -t 0 -c 0.01 -b 0 -q';

If that is still not satisfying, then you should probably use a different classifier. I would probably recommend using crossnobis (see our template) since it’s comparably fast, generally performs pretty well, and yields nice continuous results rather than binary accuracies.

Best,
Martin

MANAGED BY INCF