Multi-dataset TDT classfication question

tdt

#1

Dear Martin and all,

I used the TDT software to classify the resting-state ALFF values of the two groups. Using the searchlight method, the accuracy of the X brain sub-region was 80%, the accuracy of the Y brain sub-region was 77%. I also used the functional connectivity between the X sub-region, the Y sub-region and the whole brain voxels to classify two groups, and found that the accuracy of functional connectivity between the X and the Z brain region was 78%, the accuracy of functional connectivity between the Y and the AA brain region was 73% Then, could the TDT toolbox can combine these datasets in some way to classify two groups for achieving higher accuracy?(maybe similar to random forest or multiple results voting decisions or other methods).

Many thanks.

Yongming


#2

and another question,
when I do permutation test using demo9_permutation_analysis, the TDT asked me to input the cfg.boot.n_boot = ? , I have two groups and every group have 40 participants.
do I need to input 1 or 80?
what does n_boot mean?
when I try to input 80 and run
it said:
creating permutation designs…
design for CV decoding for 80 files X 80 steps created

did I do right?

Thanks


#3

Hi Yongming,

I think you have ensemble methods in mind, where you combine multiple different classifiers for a shared classification task (assuming you always have the same labels across all classifiers). We don’t have ensemble approaches implemented as a default, since there are a lot of different ways these could be done. However, it should be fairly easy for you to implement. Usually, a majority vote would take the most commonly voted response. Alternatively, you could take the decision values as a weighted version of the votes (which when I played around with maximizing accuracy from multiple ROIs worked better for me when there are few votes). I would ask TDT to return the decision values, i.e.

cfg.results.output = {'decision_values'}; % for a classical majority vote, use 'predicted_labels'

Then once you have all your results.mat files for the different ROIs, I would load them all in, and then combine the information across all your decoding analyses, e.g.

clear dv
load('res_region_x.mat') 
dv(:,1) = results.decision_values.output{1}; % assuming one ROI in this analysis
load('res_region_y.mat')
dv(:,2) = results.decision_values.output{1}; % same here
...

Perhaps you did it all in one analysis, then you pick results.decision_values.output{1} and results.decision_values.output{2}. Now you sum across each row of DV and take the sign to set everything to 1 or -1 (which is like a weighted majority vote):

DV = sum(dv,2);
predicted_labels = sign(DV);

Now assuming your labels were 1 and -1, you can just calculate the accuracy, i.e.

true_labels = cfg.design.label(:,1);
accuracy = mean(predicted_labels == true_labels);

Best,
Martin

P.S.: I just noticed the current version of getting decision_values and predicted_labels could be improved (it wouldn’t work with the code I wrote above). I just changed it and will upload the version that would work with the above code by tomorrow on our toolbox website.


#4

Thanks! I will try it!


#5

Dear Martin,
I try the script and it going well.
However, due to my brain mask had 19714 voxels, I got 19714 row in results.decision_values.output.
I can make a loop function to get a 19714*1 accuracy matrix.

Then how can I change this accuracy.mat  to a brain map

or suppose the accuracy of the line 10 is the highest, how can I know where its coordinates are?

many thanks!

Bests
yongming Wang


#6

Hi Yongming,

Since manually writing results is a recurring issue, I created a little help text file that explains how to do this, which I’ll upload with the next update of the toolbox. Here are the relevant parts:

What you need to to

  1. Get a cfg. You can load the original cfg you used to run TDT (usually saved as res_cfg.mat).
  2. Assume you want to call your output ensemble_accuracy. Then set the following fields:
    cfg.results.output = {'ensemble_accuracy'};
    cfg.results.resultsname = {'res_ensemble_accuracy'};
  3. Get your results struct from a previous call of TDT (usually saved as e.g. res_accuracy_minus_chance.mat).
  4. Assume again you want to call your output ensemble_accuracy. Then set the following field:
    results.ensemble_accuracy.output = <place the vector of results here>;
    (length of vector must match results.mask_index).
  5. If you want to a clean results struct, remove previously existing results, e.g.:
    results = rmfield(results,'accuracy_minus_chance');
  6. Call decoding_write_results(cfg,results);
  7. Enjoy :wink:

Example call

load('res_accuracy_minus_chance.mat')
load('res_cfg.mat')
cfg.results.output = {'ensemble_accuracy'};
cfg.results.resultsname = {'res_ensemble_accuracy'};
results.ensemble_accuracy.output = results.accuracy_minus_chance.output;
results = rmfield(results,'accuracy_minus_chance');
decoding_write_results(cfg,results);


#7

Dear Martin,

Thanks very much. It went incredibly smoothly.

bests

Yongming Wang