Hi Yongming,
I think you have ensemble methods in mind, where you combine multiple different classifiers for a shared classification task (assuming you always have the same labels across all classifiers). We don’t have ensemble approaches implemented as a default, since there are a lot of different ways these could be done. However, it should be fairly easy for you to implement. Usually, a majority vote would take the most commonly voted response. Alternatively, you could take the decision values as a weighted version of the votes (which when I played around with maximizing accuracy from multiple ROIs worked better for me when there are few votes). I would ask TDT to return the decision values, i.e.
cfg.results.output = {'decision_values'}; % for a classical majority vote, use 'predicted_labels'
Then once you have all your results.mat files for the different ROIs, I would load them all in, and then combine the information across all your decoding analyses, e.g.
clear dv
load('res_region_x.mat')
dv(:,1) = results.decision_values.output{1}; % assuming one ROI in this analysis
load('res_region_y.mat')
dv(:,2) = results.decision_values.output{1}; % same here
...
Perhaps you did it all in one analysis, then you pick results.decision_values.output{1} and results.decision_values.output{2}. Now you sum across each row of DV and take the sign to set everything to 1 or -1 (which is like a weighted majority vote):
DV = sum(dv,2);
predicted_labels = sign(DV);
Now assuming your labels were 1 and -1, you can just calculate the accuracy, i.e.
true_labels = cfg.design.label(:,1);
accuracy = mean(predicted_labels == true_labels);
Best,
Martin
P.S.: I just noticed the current version of getting decision_values and predicted_labels could be improved (it wouldn’t work with the code I wrote above). I just changed it and will upload the version that would work with the above code by tomorrow on our toolbox website.
