Random‐effects meta‐analysis models for the odds ratio in the case of rare events under different data‐generating models: A simulation study

Jansen, Katrin; Holling, Heinz

doi:10.1002/bimj.202200132

Cited by 6 publications

(14 citation statements)

References 58 publications

(126 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When meta‐analyses of rare events are based on small numbers of studies, the ML approach for the BBM may fail to converge, or it may provide biased estimates 9,10 . Bayesian estimation may be well‐suited to solve these problems, because by incorporating sensible prior information, Bayesian methods can help to stabilize parameter estimates.…”

Section: Bayesian Estimation Of the Bbmmentioning

confidence: 99%

“…As our aim was to evaluate the performance of frequentist and Bayesian BBMs for meta‐analyses of few studies, we generated meta‐analyses with five and 10 studies. Although smaller numbers of studies are common in meta‐analyses, 14,26 simulations have shown that the performance of frequentist models is already compromised when meta‐analyses consist of only five studies 9,10 . We considered conditions with small and large sample sizes (

n_{i} = 100

and

n_{i} = 500

, where

n_{i}

is the total sample size of the

i - th

study).…”

Section: Simulation Studymentioning

confidence: 99%

“…For the between‐study standard deviation, we chose the values

τ = 0.1

τ = 0.3

, and

τ = 0.6

. Values close to

τ = 0.3

are a typical choice for conditions with moderate heterogeneity in simulation studies on rare events 10,14 …”

Section: Simulation Studymentioning

confidence: 99%

“…In addition, the BBM incorporates heterogeneity by assuming that the event probabilities in both groups follow beta distributions. Several simulation studies have demonstrated that when heterogeneity is present, performance of the BBM is superior to many alternative fixed‐effects and random‐effects approaches 6,9–11 …”

Section: Introductionmentioning

confidence: 99%

“…Here, we aim to investigate whether the BBM benefits from the use of Bayesian estimation, because when meta‐analytic data are sparse, for instance when events are very rare or when there are only few studies, ML often provides biased estimates or fails to converge. Although the frequentist BBM performs well compared to alternative methods even for small numbers of studies, given that events are not too rare, 12,13 simulation studies focusing on rare events also found that its performance is unstable for as few as five studies 9,10 . A re‐analysis of Cochrane reviews by Günhan et al 14 showed that 66% of the meta‐analyses retrieved from the Cochrane library included five or less studies.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Rare events meta‐analysis using the Bayesian beta‐binomial model

Jansen,

Holling

2023

Research Synthesis Methods

View full text Add to dashboard Cite

In meta‐analyses of rare events, it can be challenging to obtain a reliable estimate of the pooled effect, in particular when the meta‐analysis is based on a small number of studies. Recent simulation studies have shown that the beta‐binomial model is a promising candidate in this situation, but have thus far only investigated its performance in a frequentist framework. In this study, we aim to make the beta‐binomial model for meta‐analysis of rare events amenable to Bayesian inference by proposing prior distributions for the effect parameter and investigating the models' robustness to different specifications of priors for the scale parameter. To evaluate the performance of Bayesian beta‐binomial models with different priors, we conducted a simulation study with two different data generating models in which we varied the size of the pooled effect, the degree of heterogeneity, the baseline probability, and the sample size. Our results show that while some caution must be exercised when using the Bayesian beta‐binomial in meta‐analyses with extremely sparse data, the use of a weakly informative prior for the effect parameter is beneficial in terms of mean bias, mean squared error, and coverage. For the scale parameter, half‐normal and exponential distributions are identified as candidate priors in meta‐analysis of rare events using the Bayesian beta‐binomial model.

show abstract

Section: Bayesian Estimation Of the Bbmmentioning

confidence: 99%

n_{i} = 100

and

n_{i} = 500

, where

n_{i}

is the total sample size of the

i - th

study).…”

Section: Simulation Studymentioning

confidence: 99%

“…For the between‐study standard deviation, we chose the values

τ = 0.1

τ = 0.3

, and

τ = 0.6

. Values close to

τ = 0.3

are a typical choice for conditions with moderate heterogeneity in simulation studies on rare events 10,14 …”

Section: Simulation Studymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Rare events meta‐analysis using the Bayesian beta‐binomial model

Jansen,

Holling

2023

Research Synthesis Methods

View full text Add to dashboard Cite

show abstract

A re‐analysis of about 60,000 sparse data meta‐analyses suggests that using an adequate method for pooling matters

Schulz,

Kramer,

Kuss

et al. 2024

Research Synthesis Methods

View full text Add to dashboard Cite

In sparse data meta‐analyses (with few trials or zero events), conventional methods may distort results. Although better‐performing one‐stage methods have become available in recent years, their implementation remains limited in practice. This study examines the impact of using conventional methods compared to one‐stage models by re‐analysing meta‐analyses from the Cochrane Database of Systematic Reviews in scenarios with zero event trials and few trials. For each scenario, we computed one‐stage methods (Generalised linear mixed model [GLMM], Beta‐binomial model [BBM], Bayesian binomial‐normal hierarchical model using a weakly informative prior [BNHM‐WIP]) and compared them with conventional methods (Peto‐Odds‐ratio [PETO], DerSimonian‐Laird method [DL] for zero event trials; DL, Paule‐Mandel [PM], Restricted maximum likelihood [REML] method for few trials). While all methods showed similar treatment effect estimates, substantial variability in statistical precision emerged. Conventional methods generally resulted in smaller confidence intervals (CIs) compared to one‐stage models in the zero event situation. In the few trials scenario, the CI lengths were widest for the BBM on average and significance often changed compared to the PM and REML, despite the relatively wide CIs of the latter. In agreement with simulations and guidelines for meta‐analyses with zero event trials, our results suggest that one‐stage models are preferable. The best model can be either selected based on the data situation or, using a method that can be used in various situations. In the few trial situation, using BBM and additionally PM or REML for sensitivity analyses appears reasonable when conservative results are desired. Overall, our results encourage careful method selection.

show abstract

Comparing various Bayesian random‐effects models for pooling randomized controlled trials with rare events

Yao,

Jia,

Mei

et al. 2024

Pharmaceutical Statistics

View full text Add to dashboard Cite

The meta‐analysis of rare events presents unique methodological challenges owing to the small number of events. Bayesian methods are often used to combine rare events data to inform decision‐making, as they can incorporate prior information and handle studies with zero events without the need for continuity corrections. However, the comparative performances of different Bayesian models in pooling rare events data are not well understood. We conducted a simulation to compare the statistical properties of four parameterizations based on the binomial‐normal hierarchical model, using two different priors for the treatment effect: weakly informative prior (WIP) and non‐informative prior (NIP), pooling randomized controlled trials with rare events using the odds ratio metric. We also considered the beta‐binomial model proposed by Kuss and the random intercept and slope generalized linear mixed models. The simulation scenarios varied based on the treatment effect, sample size ratio between the treatment and control arms, and level of heterogeneity. Performance was evaluated using median bias, root mean square error, median width of 95% credible or confidence intervals, coverage, Type I error, and empirical power. Two reviews are used to illustrate these methods. The results demonstrate that the WIP outperforms the NIP within the same model structure. Among the compared models, the model that included the treatment effect parameter in the risk model for the control arm did not perform well. Our findings confirm that rare events meta‐analysis faces the challenge of being underpowered, highlighting the importance of reporting the power of results in empirical studies.

show abstract

Random‐effects meta‐analysis models for the odds ratio in the case of rare events under different data‐generating models: A simulation study

Abstract: This article has earned an open data badge "Reproducible Research" for making publicly available the code necessary to reproduce the reported results. The results reported in this article could fully be reproduced.

Cited by 6 publications

References 58 publications

Rare events meta‐analysis using the Bayesian beta‐binomial model

Rare events meta‐analysis using the Bayesian beta‐binomial model

A re‐analysis of about 60,000 sparse data meta‐analyses suggests that using an adequate method for pooling matters

Comparing various Bayesian random‐effects models for pooling randomized controlled trials with rare events

Contact Info

Product

Resources

About