NiMARE functionalities

I am hoping to do a coordinate-based meta-analysis with Nimare. I have extracted the peak coordinates from a set of task-based fMRI studies (~110) selected from the literature. I want to pursue the following analyses and am wondering whether someone could confirm that Nimare has these functionalities (or if not, suggest alternative tools)? I have been able to find some of them in the Nimare examples but am not sure about all of these. Thanks so much!

Main analyses:

  1. Main effect across all studies and for subsets of studies (eg. for each disorder type separately)

  2. Contrasts between subsets of studies (eg. studies with participants who have disorder A vs. studies with disorder B)

→ For both I would like to do multiple comparison correction using permutation tests + thresholding of statistical maps.

Sensitivity analyses:

  1. Robustness checks using jackknife and bootstrapping. I understand these address different concerns and ideally I’d like to do both.

  2. Using the overall sample size from each study (disorder + control) vs. the sample size of the smaller of the two study groups for the Gaussian kernel.

  3. Running a balanced contrast analysis that accounts for having a different number of studies on each side of a contrast (eg. more studies exist on disorder A vs. B), such that the side with more studies is subsampled down to the number of studies on the other side and this process is repeated.

  4. Pooling all experiments from one study into a single experiment first and then running the main and contrast analyses vs. allowing all experiments to contribute to the analyses as if they were independent.

If there are any other sensitivity analyses typically done that I am not thinking of, please let me know as well!

Hi @dclb!

Here are some brief responses to your questions.

main analyses:

  1. Main effect across all studies and for subsets of studies (eg. for each disorder type separately)

For the main effect across all groups of studies I would suggest using a Conjunction Analysis. Or if you’re just pooling all the studies, that would be a regular meta-analysis (e.g., ALE, MKDA), if I’m understanding correctly.

For each disorder type separately, these would also be regular meta-analyses (e.g., ALE, MKDA)

  1. Contrasts between subsets of studies (eg. studies with participants who have disorder A vs. studies with disorder B)

For contrasts, NiMARE has a couple of options: MKDAChi2 or ALESubtraction

For both I would like to do multiple comparison correction using permutation tests + thresholding of statistical maps.

NiMARE has montecarlo permutation testing for many of the algorithms: permutation testing (montecarlo).
Thresholds to the final statistical map can be applied using numpy or through visualization using nilearn: NiMARE: Neuroimaging Meta-Analysis Research Environment — NiMARE 0.10.0 documentation

Sensitivity analyses:

  1. Robustness checks using jackknife and bootstrapping. I understand these address different concerns and ideally I’d like to do both.

NiMARE currently has jackknife implemented, and I’ll look into permutation testing, it may be valuable to implement, a previous conversation brought this up.

you can also look at focuscounter.

  1. Using the overall sample size from each study (disorder + control) vs. the sample size of the smaller of the two study groups for the Gaussian kernel.

Currently we use the combined sample size, but I think it would be fairly straightforward to offer the option to choose the smaller of the two groups.

  1. Running a balanced contrast analysis that accounts for having a different number of studies on each side of a contrast (eg. more studies exist on disorder A vs. B), such that the side with more studies is subsampled down to the number of studies on the other side and this process is repeated.

NiMARE doesn’t have a specific utility for this, but can be done using python and a loop.

  1. Pooling all experiments from one study into a single experiment first and then running the main and contrast analyses vs. allowing all experiments to contribute to the analyses as if they were independent.

We currently allow all the experiments within a study to contribute to the analyses as if they were independent (unless you use the studyset class, but that is still a work in progress): here’s the related issue: Create Transformer to combine coordinates from different experiments in the same studies · Issue #728 · neurostuff/NiMARE · GitHub, this can also be worked on.

Let me know if you want more specific advice for any of the points provided above, and for the things NiMARE currently doesn’t offer, we can work toward implementing those. I can either help you make the changes yourself on NiMARE (and you can learn to make contributions to open source software!), or if you can prioritize the features you wanted, I’ll get to them (this may be slower).

All the best!
James

Thanks so much for promptly addressing my questions about NiMARE (my bad for the incorrect capitalization!). I’m sure more questions will come up, but for now I have the following follow-up questions:

  1. I wanted to ask more about FocusCounter. It seems to be used in the function you suggested for contrast analysis with ALE (ALESubtraction) but you also suggested it as a way to do a robustness check instead of jackknife or bootstrap. Could you elaborate what FocusCounter does?

  2. I had not come across a conjunction analysis before. How is this different from a regular meta-analysis with ALE (this had been my plan)?

  3. I am a bit confused in regards to the statistical thresholding. You mentioned that NiMARE does not do thresholding and instead the user is supposed to do it afterwards. But in the ALESubtraction I see the option for “diagnostics” which uses FocusCounter. When I examine FocusCounter, I see the options for voxel_thresh and cluster_threshold. What do these do if they are not used for thresholding?

Thanks so much!

Could you give me context to this? I will need the pooled functionality. Is the Studyset class not ready yet? Or does it do something else? I am seeing this example, would I have to do this with each of the studies that have multiple experiments? Does it work if I use my own csv files with coordinates as opposed to studies from a database?

I also came across this tool that implements the pooled analysis (and the balanced contrast). Is it comparable to NiMARE when it comes to doing a cbma?

Thanks so much!

“I wanted to ask more about FocusCounter. It seems to be used in the function you suggested for contrast analysis with ALE (ALESubtraction) but you also suggested it as a way to do a robustness check instead of jackknife or bootstrap. Could you elaborate what FocusCounter does?”

FocusCounter is a descriptive diagnostic, not a re-fitting robustness procedure. In NiMARE, it takes an existing result map, identifies clusters in that map, and then counts how many reported foci from each experiment fall inside each cluster. The actual counting happens in FocusCounter._transform(), where each study’s coordinates are converted to voxel indices and matched to cluster labels. By contrast, Jackknife actually re-fits the meta-analysis N times, each time leaving one experiment out, and measures how much the statistic changes.

So the short version is:
FocusCounter tells you “which studies put peaks in this cluster?”
Jackknife tells you “how much does this cluster/statistic depend on each study?”

If you want a true robustness/influence analysis, Jackknife is the closer match.

“I had not come across a conjunction analysis before. How is this different from a regular meta-analysis with ALE (this had been my plan)?”

In NiMARE, conjunction analysis is not a new CBMA estimator. It is post-processing on already existing images. conjunction_analysis() takes two or more input images and returns the voxelwise minimum where all inputs are positive. That means it is asking: “where do these separate maps overlap?”

A regular ALE meta-analysis is different: it starts from coordinates, builds modeled-activation maps, and estimates convergence across experiments from scratch. Conjunction is for overlap between completed analyses; ALE is the actual meta-analysis model.

“I am a bit confused in regards to the statistical thresholding. You mentioned that NiMARE does not do thresholding and instead the user is supposed to do it afterwards. But in the ALESubtraction I see the option for ‘diagnostics’ which uses FocusCounter. When I examine FocusCounter, I see the options for voxel_thresh and cluster_threshold. What do these do if they are not used for thresholding?”

Those FocusCounter thresholds are for defining clusters for the diagnostic, not for changing the fitted meta-analysis itself. In Diagnostics.transform(), NiMARE takes the chosen target_image, optionally applies voxel_thresh and cluster_threshold through Nilearn’s cluster finder, and then summarizes contributions within those clusters. The default target_image is already a corrected cluster-level map, which is why voxel_thresh defaults to None there.

So:
ALESubtraction or ALE produce the statistical maps.
Correction/thresholding is a separate step.
FocusCounter can then use an already-thresholded map, or apply a simple threshold only to define which clusters to summarize.

“Could you give me context to this? I will need the pooled functionality.”

If by “pooled functionality” you mean combining multiple experiments/contrasts from the same paper so they are not treated as independent, NiMARE does have that concept, but it is not automatic during import. By default, each analysis becomes its own experiment in the Dataset; the IDs are built as study_id-analysis_id in Dataset.init(), and convert_nimads_to_dataset() converts each NIMADS analysis separately in nimare/io.py.

Pooling is handled at the Studyset level via Studyset.combine_analyses(), which merges all analyses within each study into one combined analysis.

“Is the Studyset class not ready yet? Or does it do something else?”

Studyset is ready, but it is not a generic CSV importer. It is a NIMADS container class: load/save NIMADS, merge/slice studies, optionally combine analyses, and convert to a NiMARE Dataset. So it does something useful, but specifically in the NIMADS workflow.

“I am seeing this example 1, would I have to do this with each of the studies that have multiple experiments?”

If you use Studyset.combine_analyses(), it pools all studies in that Studyset that have more than one analysis. You would not need to do them one by one. The limitation is that it is broad: it combines every multi-analysis study, not only a hand-picked subset, unless you edit/filter first.

“Does it work if I use my own csv files with coordinates as opposed to studies from a database?”

Not directly through Studyset, because Studyset expects NIMADS JSON/dicts, not arbitrary CSV. If your data start in CSV, you would need an import/conversion step first. Conceptually, pooling can still be done, but you would usually do it while building the NiMARE input structure, or convert your CSV to NIMADS first and then use Studyset.combine_analyses().

“I also came across this tool 1 that implements the pooled analysis (and the balanced contrast). Is it comparable to NiMARE when it comes to doing a cbma?”

JALE: yes, it is comparable for ALE-focused CBMA, but it is not equivalent to NiMARE.

JALE is more specialized around ALE workflows and directly exposes:
pool_experiments
standard ALE contrast
balanced ALE contrast

NiMARE is broader overall, but today it does not implement JALE’s balanced-subsampling contrast. NiMARE’s ALESubtraction is a standard group-assignment permutation contrast, and it explicitly differs from the original ALE subtraction approach by evaluating all voxels in the mask, not only voxels significant in the main effects. JALE’s docs describe a separate balanced contrast based on repeated subsampling of the larger dataset.

So the practical answer is:
If you specifically need pooled spreadsheet-style input and balanced contrast, JALE is currently more turnkey.
If you want a more general Python meta-analysis framework with ALE, MKDA/KDA, pairwise CBMA, diagnostics, and IBMA, NiMARE is the broader library.

Apologies for the long delay in response, hope this helps you and/or anyone else that comes along this post.

Best,
James