Robust Bayesian meta‐analysis: Model‐averaging across complementary publication bias adjustment methods

Bartoš, František; Maier, Maximilian; Wagenmakers, Eric‐Jan; Doucouliagos, Hristos; Stanley, T. D.

doi:10.1002/jrsm.1594

Cited by 52 publications

(88 citation statements)

References 99 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…. on heterogeneity, six weight functions and PET-PEESE publication-bias adjustment as specified in Bartoš, Maier, et al, 2021). We set the prior hypothesis probability to 0.50 for the effect size, heterogeneity, and publication bias.…”

Section: Example Reportmentioning

confidence: 99%

Adjusting for Publication Bias in JASP and R: Selection Models, PET-PEESE, and Robust Bayesian Meta-Analysis

Bartoš

Maier

Quintana

et al. 2022

Advances in Methods and Practices in Psychological Science

Self Cite

View full text Add to dashboard Cite

Meta-analyses are essential for cumulative science, but their validity can be compromised by publication bias. To mitigate the impact of publication bias, one may apply publication-bias-adjustment techniques such as precision-effect test and precision-effect estimate with standard errors (PET-PEESE) and selection models. These methods, implemented in JASP and R, allow researchers without programming experience to conduct state-of-the-art publication-bias-adjusted meta-analysis. In this tutorial, we demonstrate how to conduct a publication-bias-adjusted meta-analysis in JASP and R and interpret the results. First, we explain two frequentist bias-correction methods: PET-PEESE and selection models. Second, we introduce robust Bayesian meta-analysis, a Bayesian approach that simultaneously considers both PET-PEESE and selection models. We illustrate the methodology on an example data set, provide an instructional video ( https://bit.ly/pubbias ) and an R-markdown script ( https://osf.io/uhaew/ ), and discuss the interpretation of the results. Finally, we include concrete guidance on reporting the meta-analytic results in an academic article.

show abstract

Section: Example Reportmentioning

confidence: 99%

Adjusting for Publication Bias in JASP and R: Selection Models, PET-PEESE, and Robust Bayesian Meta-Analysis

Bartoš

Maier

Quintana

et al. 2022

Advances in Methods and Practices in Psychological Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…Critically, our re-analysis addresses limitations in both previous meta-analyses. First, the analysis of enhanced expectancies fit only one bias correction model-the trim-and-fill method (Duval & Tweedie, 1998)-and that model has been shown to result in exaggerated effect size estimates and severely inflated Type 1 error rates in the presence of publication bias and small or null effects (Bartoš et al, 2022;Carter et al, 2019). Second, although the results from multiple reporting bias models coalesced around small effect sizes (ranged from g = -.11 to g = .26) in the analysis of self-controlled practice, there are no principled reasons for preferring one estimate over another.…”

Section: Regression Modelsmentioning

confidence: 99%

“…The self-controlled practice meta-data 2 , enhanced expectancy meta-data, and the combination of both the enhanced expectancy and self-controlled practice meta-data were analyzed with a z-curve. A z-curve analysis estimates the statistical power of all studies ever conducted within a given literature, even if those studies were not reported, on the basis of the significant results that are present (Bartoš et al, 2022). That power estimate is equivalent to the expected discovery rate, that is, the expected rate of significant results for a given literature.…”

Section: Z -Curvementioning

confidence: 99%

“…The RoBMA method provides single estimates of the average effect and heterogeneity, along with Bayes factors to quantify the evidence in support of a true effect, the presence of heterogeneity, and the presence of reporting bias. Results from simulation studies and analyses of real data with known reporting bias mechanisms suggest that RoBMA is substantially more accurate and less biased than naive random effects models and also performs better than other competing reporting bias models (Bartoš et al, 2022).…”

mentioning

confidence: 97%

“…Until recently, if the results of multiple sensitivity analyses differed widely, there was no mechanism to reconcile the estimates. We now have Robust Bayesian Meta-Analysis (RoBMA) methods that apply Bayesian model averaging to allow meta-analysts to fit several plausible models (see Table 1 for descriptions) and give greater weight to the models that best account for the data (Bartoš et al, 2022;Maier et al, in-press). The RoBMA method provides single estimates of the average effect and heterogeneity, along with Bayes factors to quantify the evidence in support of a true effect, the presence of heterogeneity, and the presence of reporting bias.…”

mentioning

confidence: 99%

See 2 more Smart Citations

The combination of reporting bias and underpowered study designs has substantially exaggerated the motor learning benefits of self-controlled practice and enhanced expectancies: A meta-analysis

McKay¹,

Bacelar²,

Parma³

et al. 2022

Preprint

View full text Add to dashboard Cite

Enhanced expectancies and autonomy-support through self-controlled practice conditions form the motivation pillar of OPTIMAL theory (Wulf & Lewthwaite, 2016). The influence of these practice variables on motor learning was recently evaluated in two separate meta-analyses. Both meta-analyses found that the published literature suggested a moderate and significant benefit on motor learning; however, evidence for reporting bias was found in both literatures. Although multiple bias-corrected estimates were reported in the self-controlled meta-analysis, there was no principled way to prefer one over the other. In the enhanced expectancies meta-analysis, the trim-and-fill-technique failed to correct the estimated effects. Here, we addressed these limitations by reanalyzing the data from both meta-analyses using robust Bayesian meta-analysis methods. Our reanalysis revealed that reporting bias substantially exaggerated the benefits of these practice variables in the original meta-analyses. The true effects are instead small, uncertain, and potentially null. We also found the estimated average statistical power among all studies from the original meta-analyses was 6% (95% confidence interval [5%, 13%]). These results provide compelling and converging evidence that strongly suggests the available literature is insufficient to support the motivation pillar of OPTIMAL theory. Our results highlight the need for adequately powered experimental designs if motor learning scientists want to make evidence-based recommendations.

show abstract

Patent Ductus Arteriosus and Bronchopulmonary Dysplasia–Associated Pulmonary Hypertension

Villamor,

van Westering-Kroon,

Gonzalez-Luis

et al. 2023

JAMA Netw Open

Self Cite

View full text Add to dashboard Cite

ImportanceBronchopulmonary dysplasia (BPD) is often associated with pulmonary vascular disease and secondary pulmonary hypertension (PH). The pathogenesis of BPD-associated PH (BPD-PH) is complex and involves prenatal and postnatal factors that disrupt pulmonary vascular development, and patent ductus arteriosus (PDA) is a factor potentially associated with risk of BPD-PH that has been identified in very recent studies.ObjectiveTo explore the association of PDA with BPD-PH using a bayesian model-averaged (BMA) meta-analysis of studies.Data SourcesPubMed and Embase were searched up to April 2023. Key search terms included BPD and PH.Study SelectionStudies examining infants with gestational age 32 weeks or less and reporting data on PDA and risk of BPD-PH.Data Extraction and SynthesisThis study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses and the Meta-Analysis of Observational Studies in Epidemiology reporting guidelines. Two independent reviewers extracted data, with a third reviewer checking for accuracy and completeness. Data pooling and effect size calculations were performed by BMA.Main Outcomes and MeasuresThe primary outcome was BPD-PH. BMA was used to calculate Bayes factors (BFs). The BF10 is the ratio of the probability of the data under the alternative hypothesis (H1, association of PDA with BPD-HP) over the probability of the data under the null hypothesis (H0).ResultsA total of 32 studies (8513 infants) were included. BMA showed that the evidence in favor of H1 was weak for any PDA (BF10 = 2.90; 10 studies), moderate for hemodynamically significant PDA (BF10 = 3.77; 3 studies), and extreme for surgically ligated or catheter-occluded PDA (BF10 = 294.9; 16 studies). In contrast, the evidence in favor of H0 was weak for medically treated PDA (BF10 = 0.55; 6 studies). In addition, BMA found strong evidence in favor of H1 when prolonged exposure to PDA was analyzed as a dichotomous variable (BF10 = 11.80; 6 studies) and extreme evidence (BF10 = 113.60; 3 studies) when PDA exposure time was analyzed as a continuous variable.Conclusions and RelevanceIn this bayesian meta-analysis, the data suggest that prolonged exposure to PDA might be associated with increased risk of pulmonary vascular disease in extremely preterm infants. This highlights the need to monitor for PH in high-risk preterm infants with prolonged exposure to PDA and to incorporate PH risk into clinical decisions regarding PDA management.

show abstract

Robust Bayesian meta‐analysis: Model‐averaging across complementary publication bias adjustment methods

Cited by 52 publications

References 99 publications

Adjusting for Publication Bias in JASP and R: Selection Models, PET-PEESE, and Robust Bayesian Meta-Analysis

Adjusting for Publication Bias in JASP and R: Selection Models, PET-PEESE, and Robust Bayesian Meta-Analysis

The combination of reporting bias and underpowered study designs has substantially exaggerated the motor learning benefits of self-controlled practice and enhanced expectancies: A meta-analysis

Patent Ductus Arteriosus and Bronchopulmonary Dysplasia–Associated Pulmonary Hypertension

Contact Info

Product

Resources

About