Hi @dclb!
Here are some brief responses to your questions.
main analyses:
- Main effect across all studies and for subsets of studies (eg. for each disorder type separately)
For the main effect across all groups of studies I would suggest using a Conjunction Analysis. Or if you’re just pooling all the studies, that would be a regular meta-analysis (e.g., ALE, MKDA), if I’m understanding correctly.
For each disorder type separately, these would also be regular meta-analyses (e.g., ALE, MKDA)
- Contrasts between subsets of studies (eg. studies with participants who have disorder A vs. studies with disorder B)
For contrasts, NiMARE has a couple of options: MKDAChi2 or ALESubtraction
For both I would like to do multiple comparison correction using permutation tests + thresholding of statistical maps.
NiMARE has montecarlo permutation testing for many of the algorithms: permutation testing (montecarlo).
Thresholds to the final statistical map can be applied using numpy or through visualization using nilearn: NiMARE: Neuroimaging Meta-Analysis Research Environment — NiMARE 0.10.0 documentation
Sensitivity analyses:
- Robustness checks using jackknife and bootstrapping. I understand these address different concerns and ideally I’d like to do both.
NiMARE currently has jackknife implemented, and I’ll look into permutation testing, it may be valuable to implement, a previous conversation brought this up.
you can also look at focuscounter.
- Using the overall sample size from each study (disorder + control) vs. the sample size of the smaller of the two study groups for the Gaussian kernel.
Currently we use the combined sample size, but I think it would be fairly straightforward to offer the option to choose the smaller of the two groups.
- Running a balanced contrast analysis that accounts for having a different number of studies on each side of a contrast (eg. more studies exist on disorder A vs. B), such that the side with more studies is subsampled down to the number of studies on the other side and this process is repeated.
NiMARE doesn’t have a specific utility for this, but can be done using python and a loop.
- Pooling all experiments from one study into a single experiment first and then running the main and contrast analyses vs. allowing all experiments to contribute to the analyses as if they were independent.
We currently allow all the experiments within a study to contribute to the analyses as if they were independent (unless you use the studyset class, but that is still a work in progress): here’s the related issue: Create Transformer to combine coordinates from different experiments in the same studies · Issue #728 · neurostuff/NiMARE · GitHub, this can also be worked on.
Let me know if you want more specific advice for any of the points provided above, and for the things NiMARE currently doesn’t offer, we can work toward implementing those. I can either help you make the changes yourself on NiMARE (and you can learn to make contributions to open source software!), or if you can prioritize the features you wanted, I’ll get to them (this may be slower).
All the best!
James