Prior Choices of Between-Study Heterogeneity in Contemporary Bayesian Network Meta-analyses: an Empirical Study

Abstract

BACKGROUND: Network meta-analysis (NMA) is a popular tool to compare multiple treatments in medical research. It is frequently implemented via Bayesian methods. The prior choice of between-study heterogeneity is critical in Bayesian NMAs. This study evaluates the impact of different priors for heterogeneity on NMA results. METHODS:We identified all NMAs with binary outcomes published in The BMJ, JAMA, and The Lancet during 2010-2018, and extracted information about their prior choices for heterogeneity. Our primary analyses focused on those with publicly available full data. We re-analyzed the NMAs using 3 commonly-used non-informative priors and empirical informative log-normal priors. We obtained the posterior median odds ratios and 95% credible intervals of all comparisons, assessed the correlation among different priors, and used Bland-Altman plots to evaluate their agreement. The kappa statistic was also used to evaluate the agreement among these priors regarding statistical significance. RESULTS: Among the selected Bayesian NMAs, 52.3% did not specify the prior choice for heterogeneity, and 84.1% did not provide rationales. We re-analyzed 19 NMAs with full data available, involving 894 studies, 173 treatments, and 395,429 patients. The correlation among posterior median (log) odds ratios using different priors were generally very strong for NMAs with over 20 studies. The informative priors produced substantially narrower credible intervals than non-informative priors, especially for NMAs with few studies. Bland-Altman plots and kappa statistics indicated strong overall agreement, but this was not always the case for a specific NMA. CONCLUSIONS: Priors should be routinely reported in Bayesian NMAs. Sensitivity analyses are recommended to examine the impact of priors, especially for NMAs with relatively small sample sizes. Informative priors may produce substantially narrower credible intervals for such NMAs.

A penalization approach to random‐effects meta‐analysis

“…From the Bayesian perspective, penalizing the between-study variance is equivalent to assigning a prior distribution to the variance component 𝜏 2 or 𝜏. Meta-analysts may use informative priors based on external evidence to aid the estimation of the between-study variance, which may be particularly helpful for meta-analyses with a relatively small number of studies. 58,59…”

Predictive P-score for treatment ranking in Bayesian network meta-analysis

“…In addition to the vague priors, secondary analyses were performed for each NMA using informative priors [ 32 , 36 ]. Specifically, based on the recommendations from Turner et al [ 32 ], we used the log-normal priors LN (−2.01, 1.64 2 ) and LN (−2.13, 1.58 2 ) for the heterogeneity variances in the smoking cessation and all-grade TrAEs data, respectively.…”

“…In the current literature, many NMAs are performed using Bayesian approaches alongside frequentist ones [ 26 – 28 ]. Bayesian approaches offer additional flexibility compared to frequentist approaches, e.g., by specifying informative priors and sophisticated variance-covariance structures within multi-arm studies [ 29 – 36 ]. This article extends the P-score to a future study setting under the Bayesian framework, with the focus on NMAs with binary outcomes.…”

Theory and practice of Bayesian and frequentist frameworks for network meta-analysis

“…In general, we want the observed data to have the most influence on the posterior effect estimates and, thus, often non‐informative or minimally informative prior distributions (in which the posterior distribution is determined as completely or as minimally as possible by the observed data) are used 23 31 32. The prior distribution should be within the range of plausible values for the pooled intervention effect and, if the prior is to be non‐informative (or vague or uniform), it should be very large, which means that the distribution is essentially flat over the plausible range of values for the treatment effect 31–33. Despite the importance of prior distributions, a recent review of 44 Bayesian NMAs published in leading general medical journals found that approximately half did not specify their choice of heterogeneity prior distributions and 84% failed to provide a rationale for their selected priors 33…”

“…The prior distribution should be within the range of plausible values for the pooled intervention effect and, if the prior is to be non‐informative (or vague or uniform), it should be very large, which means that the distribution is essentially flat over the plausible range of values for the treatment effect 31–33. Despite the importance of prior distributions, a recent review of 44 Bayesian NMAs published in leading general medical journals found that approximately half did not specify their choice of heterogeneity prior distributions and 84% failed to provide a rationale for their selected priors 33…”