Sorry to ask this question. Recently, I am interested in the multivariate pattern analysis. However, as I read the tutorials on the website of nilearn, it seems that the tutorials focus on 1 subject (e.g., this example).
Is it possible to use nilearn to conduct a group-level MVPA analysis? If yes, is there any tutorial or code example?
Thank you for the kind help.
Ahoi hoi @Ming-Che_Hsieh,
thank you very much for your post and welcome to neurostars, it’s great to have you here.
Could you maybe outline a bit more what you would like to do? Searchlights, ROI decoding, etc.?
With that it’s easier for us to assess what’s possible and what’s not (disclaimer: the latter means “what’s possible but not yet integrated”).
Cheers, Peer
Hi, @PeerHerholz. Thank you for the response!
Because I am just a beginner in MVPA and nilearn, at this moment I want to focus on something basic, such as ROI decoding, at the group-level analysis.
Thank you for the reply again.
Regards, Ming-Che
Hi @Ming-Che_Hsieh ,
funny enough this is currently discussed in the nilearn repo and we hopefully start working on it soon(-ish).
In the meantime: it’s definitely possible, but there are no respective functions/tutorials available just yet. However, as nilearn integrates and builds upon scikit-learn, this can definitely be done. I did that in the past: extracting/preparing data using nilearn and running analyses in scikit-learn, basically adapting this nilearn example. I’ll tag the experts to also hopefully get their input as well: @bthirion, @tbazeille, @jeromedockes, @SylvainTakerkart (sorry folks for brashly tagging you here).
Cheers, Peer
Hello Ming Che!
I’ve had the same problem recently. In the end, I employed the inter-subject pattern analysis (ISPA) suggested by Wang et al. in on of their recent papers. They also published some code which should get you started. If you have any more questions feel free to contact me.
yes, thanks @jAchtzehn for pointing to our ISPA paper… while waiting for a solution integrated in nilearn, this should offer a solution… the code we provide (https://github.com/SylvainTakerkart/inter_subject_pattern_analysis) is for searchlight analyses, but it is easily tunable to work with ROI decoding (without the need for SnPM in that case; the permutation test can simply be executed with the sklearn function)…
I will not be able to be of any help in July and August, but don’t hesitate to beep me anyhow if needed, I’ll answer towards the end of August / beginning of September…
Hell @Ming-Che_Hsieh, I think if your usecase is not too hard and you have enough images, the tutorial you pointed to in the first is easy to adapt.
The most important change if you have already the images you want to decode is that you should use a leave-subject-out cross validation.
To do so you should make a ‘subject’ index that tell for each image to which subject it belongs and then use ‘groups’ = ‘subjects’ instead of ‘groups’=‘sessions’ (8.3.8.6)
Ahoi hoi @tbazeille,
thank you for your input here!
That was exactly what I did here. Back then I used a GroupShuffleSplit to generate splits assigning 20% of the data to the test set per CV fold. I think it would be great to address different CV schemes in this context, especially given research work like e.g. Varoquaux et al. (2017). What’s your take on that?
@Ming-Che_Hsieh, following @tbazeille’s point, you could also change cross_val_score to
the permutation_test_score function (also mentioned by @SylvainTakerkart) to assess the significance.
Very interesting discussion, thx folks!
@SylvainTakerkart, thank you very much. I would read your paper first and try the code you kindly provided.
@PeerHerholz and @tbazeille, thank you so much. I will try, but I guess I may face some questions again…Once this really occur, could you share your ideas with me if you have leisure time? Thanks again.
Hi @Ming-Che_Hsieh,
sure thing, would be very happy and interested to hear about your further endeavors! It also helps the developers to further improve the tools and corresponding documentation they create, as well as other folks who face similar problems/have similar questions. That’s what this forum is for: go community!
In case you post again/follow up here: please make sure to include as much detail about your desing/analyses plan as possible, also including code snippets. That will make it easier for us and other folks here to understand what’s going on and help out.
Cheers, Peer
Hi everyone, we are continuing this conversation because we are trying to implement the code provided in Wang et al. 2020 using Python only (we have no access to Matlab and SnPM). We are having difficulties with the permutation test that was originally done in SnPM.
In our case, we are trying to implement the exact searchlight ISPA analysis as in the paper.
We explored the sklearn function, but we are not sure how to use it within the searchlight analysis and ISPA. For example, the Wang et al. 2020 script outputs one accuracy map per fold (subject). Should we run the permutation test for each accuracy map separately? If so, how can we achieve a “final” group result?
Or should we shuffle labels before each searchlight analysis?
@SylvainTakerkart, by any chance, have you ported the original code to Python/Nilearn? Any idea on how could we do it?
Thank you very much for your help!
Hi @tbortolini ! No progress on our side, sorry… And I havn’t followed the recent developments in nilearn to know whether a full-brain-map permutation test (similar to what’s possible in SnPM) has been made possible… Maybe the nilearn team could tell us something about this?
Dear @SylvainTakerkart, thank you very much for your reply! We couldn’t find yet a similar implementation of SnPM with Nilearn, let’s see whether someone points to a solution.
One thing that we considered, instead, would be to shuffle labels before each searchlight analysis (for each fold). Do you think that would be correct?
Hi!
for nilearn / stats / permutation, you could directly open an issue on their github to see what the developers answer: Issues · nilearn/nilearn · GitHub
for another permutation-based scheme for searchlight analysis, this paper presents and studies something based on what you mention; so yes, it’s totally valid (but computationally way more expensive)
Cheers, Sylvain
Hi Sylvain, thank you for your reply!
We’ll dig further into this.
I’ll post here any developments we have.
All the best, Tiago
MANAGED BY INCF