On Estimating Residual Heterogeneity in Random-Effects Meta-Regression: A Comparative Study

Abstract

We consider six different estimators of residual heterogeneity in random-effects meta-regression, five estimators already known and implemented in the R package metafor and one estimator not yet considered in random-effects meta-regression. In a numerical study, we investigate the properties of these residual heterogeneity estimators as well as the impact of these estimators on the properties of the regression parameter estimates. It turns out that the new estimator performs quite well in terms of bias and mean squared error. The impact of the different residual heterogeneity estimators on the actual confidence coefficient of confidence intervals for regression parameters can be substantially different as shown in the numerical study.

Comparative performance of heterogeneity variance estimators in meta‐analysis: a review of simulation studies

“…In both binary and continuous meta‐analyses, study sample sizes were most commonly generated from a uniform distribution (Sidik and Jonkman, ; Sidik and Jonkman, ; Panityakul et al . ; Novianti et al ., ). The within‐study variance of each study was then derived from these sample sizes.…”

“…Performance of the DL estimator is documented in all 12 publications, which generally suggest that DL is negatively biased when the level of heterogeneity is moderate to high. The negative bias is more prominent when within‐study variance estimates are imprecise, such as in meta‐analyses of SMDs with small study sample sizes (Malzahn et al ., ) and in binary outcome meta‐analyses (Sidik and Jonkman, ; Panityakul et al ., ), particularly when there are few events occurring in each study (Bhaumik et al ., ). Minimal negative bias was observed in continuous outcome meta‐analyses with moderate study sample sizes (Viechtbauer, ; Novianti et al ., ) and binary outcome meta‐analyses with large study samples sizes (Knapp and Hartung, ).…”

“…The estimator has a small positive bias under most simulated conditions. CA remains positively biased even when the level of heterogeneity is high, unlike many other estimators such as DL, PM and REML (Panityakul et al ., ; Novianti et al ., ). However, Chung et al .…”

Methods for calculating confidence and credible intervals for the residual between-study variance in random effects meta-regression models

“…To check numerically that all the proposed methods for calculating the proposed extension of the Paule-Mandel estimator agree, this was obtained in three ways: using the metafor package and the empirical Bayes option, using the code provided in the Appendix of Panityakul et al [12], and also using the proposed Newton Raphson procedure with c =14. All three methods gave ${\hat{\tau }}_{\text{italicPM}}^2=0.219$.…”

“…This idea can be generalised to the meta-regression scenario considered here [12] by solving $Q\left({\hat{\tau }}_{\text{italicPM}}^2\right)=n-p$. This equation can be solved using the Newton Raphson procedure suggested above, with c = n - p , and if there is no solution to this equation then ${\hat{\tau }}_{\text{italicPM}}^2=0$[13,22].…”