Skip to content

Commit

Permalink
Disable computation of probabilities by default for MKDAChi2 (#856)
Browse files Browse the repository at this point in the history
* Disable computation of probabilites by default, and rename specificity to association

* Only compute certain probabilities if prior is explicitly given

* Give pFgA by default

* Remove compute probabilities variable

* Fix typo

* update tuturial to taylors' comments

* lint

* Update nimare/meta/cbma/mkda.py

Co-authored-by: James Kent <[email protected]>

* Output adjusted maps only if prior is given

* Fix typo

* Run Black

* Rename all instances to uniformity and association

* Update names in example

* Update nimare/meta/cbma/mkda.py

Co-authored-by: James Kent <[email protected]>

* Resotore notebook

* fix other prior if statement

---------

Co-authored-by: James Kent <[email protected]>
  • Loading branch information
adelavega and jdkent committed Jan 4, 2024
1 parent 0e10f03 commit efae75e
Show file tree
Hide file tree
Showing 11 changed files with 116 additions and 109 deletions.
16 changes: 8 additions & 8 deletions docs/decoding.rst
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ The correlation-based decoding is implemented in NiMARE's `CorrelationDecoder` c
decoder = CorrelationDecoder(
frequency_threshold=0.001,
meta_estimator=mkda.MKDAChi2,
target_image='z_desc-specificity',
target_image='z_desc-association',
)
decoder.fit(ns_dset)
decoding_results = decoder.transform('pain_map.nii.gz')
Expand Down Expand Up @@ -140,7 +140,7 @@ of the database in a confusion matrix. For each label in the ontology, studies a
four groups: selected and label-positive (:math:`S_{s+l+}`), selected and label-negative
(:math:`S_{s+l-}`), unselected and label-positive (:math:`S_{s-l+}`), and unselected and
label-negative (:math:`S_{s-l-}`). Each method then compares these groups in order to evaluate
both consistency and specificity of the relationship between the selection criteria and each label,
both uniformity and association of the relationship between the selection criteria and each label,
which are evaluated in terms of both statistical significance and effect size.


Expand All @@ -162,10 +162,10 @@ individual peak, and so weighting based on the number of foci would be inappropr

This decoding method produces four outputs for each label. First, the distribution of studies in
the sample with the label are compared to the distributions of other labels within the sample.
This consistency analysis produces both a measure of statistical significance (i.e., a p-value)
This uniformity analysis produces both a measure of statistical significance (i.e., a p-value)
and a measure of effect size (i.e., the likelihood of being selected given the presence of the
label). Next, the studies in the sample are compared to the studies in the rest of the database.
This specificity analysis produces a p-value and an effect size measure of the posterior
This association analysis produces a p-value and an effect size measure of the posterior
probability of having the label given selection into the sample. A detailed algorithm description
is presented below.

Expand Down Expand Up @@ -261,17 +261,17 @@ those in a larger database, but, unlike the BrainMap method, does not take foci
this reason, the Neurosynth method would likely be more appropriate for selection criteria not
based on regions of interest (e.g., for characterizing meta-analytic groupings from a
meta-analytic clustering analysis). However, the Neurosynth method requires user-provided
information that BrainMap does not. Namely, in order to estimate probabilities for the consistency
and specificity analyses with Bayes' Theorem, the Neurosynth method requires a prior probability of
information that BrainMap does not. Namely, in order to estimate probabilities for the uniformity
and association analyses with Bayes' Theorem, the Neurosynth method requires a prior probability of
a given label. Typically, a value of 0.5 is used (i.e., the estimated probability that an
individual is undergoing a given mental process described by a label, barring any evidence from
neuroimaging data, is predicted to be 50%). This is, admittedly, a poor prediction, which means
that probabilities estimated based on this prior are not likely to be accurate, though they may
still serve as useful estimates of effect size for the analysis.

Like the BrainMap method, this method produces four outputs for each label. For the consistency
Like the BrainMap method, this method produces four outputs for each label. For the uniformity
analysis, this method produces both a p-value and a conditional probability of selection given the
presence of the label and the prior probability of having the label. For the specificity analysis,
presence of the label and the prior probability of having the label. For the association analysis,
the Neurosynth method produces both a p-value and a posterior probability of presence of the label
given selection and the prior probability of having the label. A detailed algorithm description is
presented below.
Expand Down
4 changes: 2 additions & 2 deletions examples/02_meta-analyses/01_plot_cbma.py
Original file line number Diff line number Diff line change
Expand Up @@ -84,13 +84,13 @@
cres = corr.transform(results)

plot_stat_map(
results.get_map("z_desc-consistency"),
results.get_map("z_desc-uniformity"),
draw_cross=False,
cmap="RdBu_r",
threshold=0.1,
)
plot_stat_map(
cres.get_map("z_desc-consistencySize_level-cluster_corr-FWE_method-montecarlo"),
cres.get_map("z_desc-uniformitySize_level-cluster_corr-FWE_method-montecarlo"),
draw_cross=False,
cmap="RdBu_r",
threshold=0.1,
Expand Down
4 changes: 2 additions & 2 deletions examples/02_meta-analyses/07_macm.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,9 +65,9 @@
corr = FWECorrector(method="montecarlo", n_iters=10000)
cres = corr.transform(results)

# We want the "specificity" map (2-way chi-square between sel and unsel)
# We want the "association" map (2-way chi-square between sel and unsel)
plotting.plot_stat_map(
cres.get_map("z_desc-consistency_level-voxel_corr-FWE_method-montecarlo"),
cres.get_map("z_desc-uniformity_level-voxel_corr-FWE_method-montecarlo"),
threshold=3.09,
draw_cross=False,
cmap="RdBu_r",
Expand Down
2 changes: 1 addition & 1 deletion examples/02_meta-analyses/11_plot_cbmr.py
Original file line number Diff line number Diff line change
Expand Up @@ -357,7 +357,7 @@

###############################################################################
# Now that we have done group comparison tests with the specified contrast matrix,
# we can plot the z-score maps indicating consistency in activation regions among
# we can plot the z-score maps indicating uniformity in activation regions among
# all four groups.

plot_stat_map(
Expand Down
2 changes: 1 addition & 1 deletion nimare/decode/continuous.py
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,7 @@ def __init__(
features=None,
frequency_threshold=0.001,
meta_estimator=None,
target_image="z_desc-specificity",
target_image="z_desc-association",
n_cores=1,
):
meta_estimator = (
Expand Down
6 changes: 3 additions & 3 deletions nimare/decode/discrete.py
Original file line number Diff line number Diff line change
Expand Up @@ -325,7 +325,7 @@ def brainmap_decode(
n_selected_term - np.mean(n_selected_term)
).ravel() # pylint: disable=no-member

# Two-way chi-square test for specificity of activation
# Two-way chi-square test for association of activation
cells = np.array(
[
[n_selected_term, n_selected_noterm], # pylint: disable=no-member
Expand Down Expand Up @@ -605,14 +605,14 @@ def neurosynth_decode(
prior = p_term

# Significance testing
# One-way chi-square test for consistency of term frequency across terms
# One-way chi-square test for uniformity of term frequency across terms
chi2_fi = one_way(n_selected_term, n_term)
p_fi = special.chdtrc(1, chi2_fi)
sign_fi = np.sign(
n_selected_term - np.mean(n_selected_term)
).ravel() # pylint: disable=no-member

# Two-way chi-square test for specificity of activation
# Two-way chi-square test for association
cells = np.array(
[
[n_selected_term, n_selected_noterm], # pylint: disable=no-member
Expand Down
4 changes: 2 additions & 2 deletions nimare/diagnostics.py
Original file line number Diff line number Diff line change
Expand Up @@ -310,9 +310,9 @@ def _transform(self, expid, label_map, sign, result):
cluster_masker.fit(label_map)

# CBMAs have "stat" maps, while most IBMAs have "est" maps. ALESubtraction has
# stat_desc-group1MinusGroup2" maps, while MKDAChi2 has "z_desc-specificity" maps.
# stat_desc-group1MinusGroup2" maps, while MKDAChi2 has "z_desc-association' maps.
# Fisher's and Stouffer's only have "z" maps though.
target_value_keys = {"stat", "est", "stat_desc-group1MinusGroup2", "z_desc-specificity"}
target_value_keys = {"stat", "est", "stat_desc-group1MinusGroup2", "z_desc-association"}
avail_value_keys = set(result.maps.keys())
union_value_keys = list(target_value_keys & avail_value_keys)
target_value_map = union_value_keys[0] if union_value_keys else "z"
Expand Down
4 changes: 2 additions & 2 deletions nimare/meta/cbma/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -372,7 +372,7 @@ def _p_to_summarystat(self, p, null_method=None):
null_distribution /= np.max(null_distribution)
null_distribution = np.squeeze(null_distribution)

# Desired bin is the first one _before_ the target p-value (for consistency
# Desired bin is the first one _before_ the target p-value (for uniformity
# with the montecarlo null).
ss_idx = np.maximum(0, np.where(null_distribution <= p)[0][0] - 1)
ss = self.null_distributions_["histogram_bins"][ss_idx]
Expand All @@ -382,7 +382,7 @@ def _p_to_summarystat(self, p, null_method=None):
assert "histweights_corr-none_method-montecarlo" in self.null_distributions_.keys()

hist_weights = self.null_distributions_["histweights_corr-none_method-montecarlo"]
# Desired bin is the first one _before_ the target p-value (for consistency
# Desired bin is the first one _before_ the target p-value (for uniformity
# with the montecarlo null).
ss_idx = np.maximum(0, np.where(hist_weights <= p)[0][0] - 1)
ss = self.null_distributions_["histogram_bins"][ss_idx]
Expand Down
Loading

0 comments on commit efae75e

Please sign in to comment.