Robust Bayesian meta-analysis: Addressing publication bias with model-averaging.

Maier, Maximilian; Bartoš, František; Wagenmakers, Eric‐Jan

doi:10.1037/met0000405

Cited by 61 publications

(82 citation statements)

References 71 publications

(136 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A newly proposed bias correction technique, robust Bayesian metaanalysis (RoBMA) ( 6 ), avoids an all-or-none debate over whether or not publication bias is “severe.” RoBMA simultaneously applies 1) selection models that estimate relative publication probabilities ( 7 ) and 2) models of the relationship between effect sizes and SEs [i.e., Precision Effect Test and Precision Effect Estimate with Standard Error ( 6 , 8 , 9 )]. Multimodel inference is then guided mostly by those models that predict the observed data best ( 6 , 9 , 10 ). RoBMA makes multimodel inferences about the presence or absence of an effect, heterogeneity, and publication bias ( 6 , 9 ).…”

mentioning

confidence: 99%

See 1 more Smart Citation

No evidence for nudging after adjusting for publication bias

Maier

Bartoš

Stanley

et al. 2022

Proc. Natl. Acad. Sci. U.S.A.

165

View full text Add to dashboard Cite

mentioning

confidence: 99%

“…Multimodel inference is then guided mostly by those models that predict the observed data best ( 6 , 9 , 10 ). RoBMA makes multimodel inferences about the presence or absence of an effect, heterogeneity, and publication bias ( 6 , 9 ).…”

mentioning

confidence: 99%

No evidence for nudging after adjusting for publication bias

Maier

Bartoš

Stanley

et al. 2022

Proc. Natl. Acad. Sci. U.S.A.

165

View full text Add to dashboard Cite

“…Further, we conduct exploratory Bayesian [ 52 , 53 ] where we use Bayesian linear regression analyses that draw on Bayesian model averaging [ 54 , 55 ]. First, in Table 7 , we report results where we compare the individual models with one predictor each (the respective explanatory variable) against the null model with an intercept only.…”

Section: Resultsmentioning

confidence: 99%

Sure-thing vs. probabilistic charitable giving: Experimental evidence on the role of individual differences in risky and ambiguous charitable decision-making

Schönegger

Costa‐Gomes

2022

PLoS ONE

View full text Add to dashboard Cite

Charities differ, among other things, alongside the likelihood that their interventions succeed and produce the desired outcomes and alongside the extent that such likelihood can even be articulated numerically. In this paper, we investigate what best explains charitable giving behaviour regarding charities that have interventions that will succeed with a quantifiable and high probability (sure-thing charities) and charities that have interventions that only have a small and hard to quantify probability of bringing about the desired end (probabilistic charities). We study individual differences in risk/ambiguity attitudes, empathy, numeracy, optimism, and donor type (warm glow vs. pure altruistic donor type) as potential predictors of this choice. We conduct a money incentivised, pre-registered experiment on Prolific on a representative UK sample (n = 1,506) to investigate participant choices (i) between these two types of charities and (ii) about one randomly selected charity. Overall, we find little to no evidence that individual differences predict choices regarding decisions about sure-thing and probabilistic charities, with the exception that a purely altruistic donor type predicts donations to probabilistic charities when participants were presented with a randomly selected charity in (ii). Conducting exploratory equivalence tests, we find that the data provide robust evidence in favour of the absence of an effect (or a negligibly small effect) where we fail to reject the null. This is corroborated by exploratory Bayesian analyses. We take this paper to be contributing to the literature on charitable giving via this comprehensive null-result in pursuit of contributing to a cumulative science.

show abstract

“…1 In either of these cases, publication bias might not induce asymmetry in the funnel plot. Besides their assumptions regarding the publication process, funnel plot methods can perform poorly when effects are heterogeneous across studies, as we detail below (e.g., Carter et al, 2019;Maier et al, 2022). For these reasons, funnel plot methods should be supplemented with selection models, or other comparable methods, so as to more effectively assess publication bias.…”

Section: The Need For Other Methods: Uses and Limitations Of Funnel P...mentioning

confidence: 99%

Using selection models to assess sensitivity to publication bias: A tutorial and call for more routine use

Maier

VanderWeele

Mathur

2022

Campbell Systematic Reviews

Self Cite

View full text Add to dashboard Cite

In meta-analyses, it is critical to assess the extent to which publication bias might have compromised the results. Classical methods based on the funnel plot, including Egger's test and Trim-and-Fill, have become the de facto default methods to do so, with a large majority of recent meta-analyses in top medical journals (85%) assessing for publication bias exclusively using these methods. However, these classical funnel plot methods have important limitations when used as the sole means of assessing publication bias: they essentially assume that the publication process favors large point estimates for small studies and does not affect the largest studies, and they can perform poorly when effects are heterogeneous. In light of these limitations, we recommend that meta-analyses routinely apply other publication bias methods in addition to or instead of classical funnel plot methods. To this end, we describe how to use and interpret selection models. These methods make the often more realistic assumption that publication bias favors "statistically significant" results and that also directly accommodate effect heterogeneity.Selection models are well-established in the statistics literature and are supported by user-friendly software, yet remain rarely reported in many disciplines. We use previously published meta-analyses to demonstrate that selection models can yield insights that extend beyond those provided by funnel plot methods, suggesting the importance of establishing more comprehensive reporting practices for publication bias assessment.

show abstract

Robust Bayesian meta-analysis: Addressing publication bias with model-averaging.

Cited by 61 publications

References 71 publications

No evidence for nudging after adjusting for publication bias

No evidence for nudging after adjusting for publication bias

Sure-thing vs. probabilistic charitable giving: Experimental evidence on the role of individual differences in risky and ambiguous charitable decision-making

Using selection models to assess sensitivity to publication bias: A tutorial and call for more routine use

Contact Info

Product

Resources

About