TDT cross-classification analysis

1

Dear Martin,

In my study, I have two conditions (‘real’ and ‘control’) and I am modelling my trials separately as I have a low number of runs. I would like to run a cross-classification, where I train the classifier on the trials from the first run and would like to evaluate the performance of this classifier on the data from run2 and run3. I have a different number of trials in run1 than in run2 & 3 and when I try to run make_design_xclass_cv.m I get one of the follwing two errors:

  1. “Error using make_design_xclass_cv (line 229)
    Wrong number of labels in cfg.files.xclass. Cross classification needs exactly one training and one test set.”

In this case, I find that cfg.files.xclass contains “1” in every cell for some reason.

In this case, the dimensions of cfg.files.descr are 1x84, while name, chunk and label are all 84x1. Can this be part of the problem? In this one, I have entries which look fine to me, like Sn(1) real_1*bf(1) and so on.
If I create the cfg.files.xclass matrix manually based on the info in cfg.files.descr , I get the following error:

  1. “Error using decoding_basic_checks (line 235)
    Number of used labels varies across decoding steps which prevents comparing results across steps. If multiple sets are used, run them separately.”

I am a bit confused about the terminology of the error message here I think. I have two labels, real and control, am I correct? Is the error referring to the issue that I have different number of trials in the train and test dataset? How can I overcome this? I was also thinking about setting the number of steps manually, but I am not sure whether this is a good idea and If so, how can I set this.

Thanks a lot,
Dora

2

I just found this post: https://neurostars.org/t/training-and-testing-with-different-data-sets/3767 which deals with the same issue

3

Using xclass instead of xclass_cv seemingly helped, however I still have to create cfg.files.xclass matrix manually, should I worry about this?

4

Hi Dora,

I think under normal circumstances there should not be an issue with this.

The following code is from the template and I just adapted it for what you need.

The following extracts regressor names from the SPM.mat at path beta_loc:
regressor_names = design_from_spm(beta_loc);
The following maps regressors with names labelname1, labelname2 etc. to cfg.files:
cfg = decoding_describe_data(cfg,{labelname1 labelname2 labelname3 labelname4},[1 -1 1 -1],regressor_names,beta_loc,[1 1 2 2]);

The last entry maps the four labels to cross-classification splits (i.e. train on labelname1 and labelname2, test on labelname3 and labelname4, and possibly vice-verse if you set twoway = 1 in make_design_xclass).

I don’t understand why make_design_xclass_cv doesn’t work, though, but based on your description it sounds as if you want to use make_design_xclass anyway.

Hope that solves everything!
Martin

5

Hi Martin!

Thank you, it worked perfectly! I went south with the last entry, I had the misconception that it had to do something with the run/chunk numbers for some reason… Thanks again!

Dora