A penalization approach to random‐effects meta‐analysis

Abstract

Systematic reviews and meta-analyses are principal tools to synthesize evidence from multiple independent sources in many research fields. The assessment of heterogeneity among collected studies is a critical step when performing a meta-analysis, given its influence on model selection and conclusions about treatment effects. A common-effect (CE) model is conventionally used when the studies are deemed homogeneous, while a random-effects (RE) model is used for heterogeneous studies. However, both models have limitations. For example, the CE model produces excessively conservative confidence intervals with low coverage probabilities when the collected studies have heterogeneous treatment effects. The RE model, on the other hand, assigns higher weights to small studies compared to the CE model. In the presence of small-study effects or publication bias, the over-weighted small studies from a RE model can lead to substantially biased overall treatment effect estimates. In addition, outlying studies may exaggerate between-study heterogeneity. This article introduces penalization methods as a compromise between the CE and RE models. The proposed methods are motivated by the penalized likelihood approach, which is widely used in the current literature to control model complexity and reduce variances of parameter estimates. We compare the existing and proposed methods with simulated data and several case studies to illustrate the benefits of the penalization methods.

Comparisons of various estimates of the I2 statistic for quantifying between-study heterogeneity in meta-analysis

“…is the pooled CE estimate of the overall effect size 𝜇. It follows a 𝜒 2 k−1 distribution under the null hypothesis. Under the RE model, using the between-study variance estimate τ2 , the overall mean effect size is estimated as…”

“…Assessing heterogeneity between the collected studies is a critical step to examine whether the studies may be properly combined and the synthesized results are reliable. 1,2 In this article, heterogeneity refers to the variation in underlying treatment effects across studies. 3 A classical method to detect heterogeneity is the chi-squared Q test; the distribution of the Q statistic is approximately 𝜒 2 k−1 (k is the number of studies) under the null hypothesis that all studies in a meta-analysis are homogeneous.…”

“…1,2 In this article, heterogeneity refers to the variation in underlying treatment effects across studies. 3 A classical method to detect heterogeneity is the chi-squared Q test; the distribution of the Q statistic is approximately 𝜒 2 k−1 (k is the number of studies) under the null hypothesis that all studies in a meta-analysis are homogeneous. However, the Q test alone does not suffice to describe the amount of heterogeneity because only P-values are produced to indicate a binary decision of either the presence or absence of heterogeneity.…”

“…The I 2 statistic has been a popular alternative to quantify heterogeneity because of its attractive interpretation as the proportion of total variation caused by heterogeneity rather than within-study sampling error. 4,5 Specifically, the I 2 statistic can be conceptualized as 𝜏 2 ∕(𝜏 2 + 𝜎 2 ) × 100%, where 𝜏 2 is the between-study variance caused by heterogeneity and 𝜎 2 is a summary of within-study variances. It ranges from 0% to 100%.…”

“…10,11 The sample sizes within individual studies can inflate or deflate I 2 under different circumstances. 11,12 Moreover, I 2 inherits the following misunderstanding about the distribution of the Q statistic under the null hypothesis: the 𝜒 2 k−1 approximation holds for large within-study sample sizes, but it is not accurate for small and moderate sample sizes. 13,14 In response to the above shortcomings, the associated 95% confidence interval (CI) should be reported to accompany the I 2 statistic.…”