A Tutorial on Bayesian Model-Averaged Meta-Analysis in JASP

Berkhout, Sophie W.; Haaf, Julia M.; Gronau, Quentin Frederik; Heck, Daniel W.; Wagenmakers, Eric‐Jan

doi:10.31234/osf.io/ne8dw

Cited by 4 publications

(2 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To examine the robustness of the evidence for catastrophic set size limits in infants, we conducted a Bayesian meta‐analysis. Using a Bayesian approach to meta‐analysis is ideal for infant data: it is robust to low n and can quantify the strength of the null results by giving the odds of the null hypothesis over the alternative given the extant data (Bartoš et al., 2023; Berkhout et al., 2023). The goal is to use all of the evidence from the literature to estimate the extent to which the evidence points toward the null hypothesis that infants’ object representational capacities fail “catastrophically” when they are tasked with tracking four or more objects in a location.…”

Section: Bayesian Meta‐analysismentioning

confidence: 99%

“Catastrophic” set size limits on infants’ capacity to represent objects: A systematic review and Bayesian meta‐analysis

Wang,

Kibbe

2024

Developmental Science

View full text Add to dashboard Cite

Decades of research has revealed that humans can concurrently represent small quantities of three‐dimensional objects as those objects move through space or into occlusion. For infants (but not older children or adults), this ability apparently comes with a significant limitation: when the number of occluded objects exceeds three, infants experience what has been characterized as a “catastrophic” set size limit, failing to represent even the approximate quantity of the hidden array. Infants’ apparent catastrophic representational failures suggest a significant information processing limitation in the first years of life, and the evidence has been used as support for prominent theories of the development of object and numerical cognition. However, the evidence for catastrophic failure consists of individual small‐n experiments that use null hypothesis significance testing to obtain null results (i.e., p > 0.05). Whether catastrophic representational failures are robust or reliable across studies, methods, and labs is not known. Here we report a systematic review and Bayesian meta‐analysis to examine the strength of the evidence in favor of catastrophic representational failures in infancy. Our analysis of 22 experiments across 12 reports, with a combined total of n = 367 infants aged 10–20 months, revealed strong support for the evidence for catastrophic set size limits. A complementary analysis found moderate support for infants’ success when representing fewer than four objects. We discuss the implications of our findings for theories of object and numerical cognitive development.Research Highlights Previous work has suggested that infants are unable to concurrently represent four or more objects—a “catastrophic” set size limit. We reviewed this work and conducted a Bayesian meta‐analysis to examine the robustness of this limit across individual small‐n experiments. We found strong support for the evidence for catastrophic set size limits, and moderate support for infants’ success when representing fewer than four objects.

show abstract

Section: Bayesian Meta‐analysismentioning

confidence: 99%

“Catastrophic” set size limits on infants’ capacity to represent objects: A systematic review and Bayesian meta‐analysis

Wang,

Kibbe

2024

Developmental Science

View full text Add to dashboard Cite

show abstract

“…In the presence of heterogeneity, moderators (e.g., the type of experimental setup) can be included into a metaregression model to explain the effect-size heterogeneity. An interesting proposal to combine evidence from these different models into a single analysis is called Bayesian model-averaged meta-analysis (Berkhout et al, 2023; Gronau et al, 2021). However, this model is beyond the scope of the tutorial.…”

Section: Meta-analysis Introductionmentioning

confidence: 99%

Understanding Meta-Analysis Through Data Simulation With Applications to Power Analysis

Gambarota,

Altoè

2024

Advances in Methods and Practices in Psychological Science

View full text Add to dashboard Cite

Meta-analysis is a powerful tool to combine evidence from existing literature. Despite several introductory and advanced materials about organizing, conducting, and reporting a meta-analysis, to our knowledge, there are no introductive materials about simulating the most common meta-analysis models. Data simulation is essential for developing and validating new statistical models and procedures. Furthermore, data simulation is a powerful educational tool for understanding a statistical method. In this tutorial, we show how to simulate equal-effects, random-effects, and metaregression models and illustrate how to estimate statistical power. Simulations for multilevel and multivariate models are available in the Supplemental Material available online. All materials associated with this article can be accessed on OSF ( https://osf.io/54djn/ ).

show abstract

Extracorporeal cardiopulmonary resuscitation versus standard treatment for refractory out-of-hospital cardiac arrest: a Bayesian meta-analysis

Heuts,

Ubben,

Kawczynski

et al. 2024

Crit Care

View full text Add to dashboard Cite

Background The outcomes of several randomized trials on extracorporeal cardiopulmonary resuscitation (ECPR) in patients with refractory out-of-hospital cardiac arrest were examined using frequentist methods, resulting in a dichotomous interpretation of results based on p-values rather than in the probability of clinically relevant treatment effects. To determine such a probability of a clinically relevant ECPR-based treatment effect on neurological outcomes, the authors of these trials performed a Bayesian meta-analysis of the totality of randomized ECPR evidence. Methods A systematic search was applied to three electronic databases. Randomized trials that compared ECPR-based treatment with conventional CPR for refractory out-of-hospital cardiac arrest were included. The study was preregistered in INPLASY (INPLASY2023120060). The primary Bayesian hierarchical meta-analysis estimated the difference in 6-month neurologically favorable survival in patients with all rhythms, and a secondary analysis assessed this difference in patients with shockable rhythms (Bayesian hierarchical random-effects model). Primary Bayesian analyses were performed under vague priors. Outcomes were formulated as estimated median relative risks, mean absolute risk differences, and numbers needed to treat with corresponding 95% credible intervals (CrIs). The posterior probabilities of various clinically relevant absolute risk difference thresholds were estimated. Results Three randomized trials were included in the analysis (ECPR, n = 209 patients; conventional CPR, n = 211 patients). The estimated median relative risk of ECPR for 6-month neurologically favorable survival was 1.47 (95%CrI 0.73–3.32) with a mean absolute risk difference of 8.7% (− 5.0; 42.7%) in patients with all rhythms, and the median relative risk was 1.54 (95%CrI 0.79–3.71) with a mean absolute risk difference of 10.8% (95%CrI − 4.2; 73.9%) in patients with shockable rhythms. The posterior probabilities of an absolute risk difference > 0% and > 5% were 91.0% and 71.1% in patients with all rhythms and 92.4% and 75.8% in patients with shockable rhythms, respectively. Conclusion The current Bayesian meta-analysis found a 71.1% and 75.8% posterior probability of a clinically relevant ECPR-based treatment effect on 6-month neurologically favorable survival in patients with all rhythms and shockable rhythms. These results must be interpreted within the context of the reported credible intervals and varying designs of the randomized trials. Registration INPLASY (INPLASY2023120060, December 14th, 2023, https://doi.org/10.37766/inplasy2023.12.0060).

show abstract

A Tutorial on Bayesian Model-Averaged Meta-Analysis in JASP

Cited by 4 publications

References 23 publications

“Catastrophic” set size limits on infants’ capacity to represent objects: A systematic review and Bayesian meta‐analysis

“Catastrophic” set size limits on infants’ capacity to represent objects: A systematic review and Bayesian meta‐analysis

Understanding Meta-Analysis Through Data Simulation With Applications to Power Analysis

Extracorporeal cardiopulmonary resuscitation versus standard treatment for refractory out-of-hospital cardiac arrest: a Bayesian meta-analysis

Contact Info

Product

Resources

About