A simulation study comparing properties of heterogeneity measures in meta‐analyses

Abstract

The assessment of heterogeneity or between-study variance is an important issue in meta-analysis. It determines the statistical methods to be used and the interpretation of the results. Tests of heterogeneity may be misleading either due to low power for sparse data or to the detection of irrelevant amounts of heterogeneity when many studies are involved. In the former case, notable heterogeneity may remain unconsidered and an unsuitable model may be chosen and the latter case may lead to unnecessary complex analyses strategies. Measures of heterogeneity are better suited to determine appropriate analyses strategies. We review two measures with different scaling and compare them with the heterogeneity test. Estimates of the within-study variance are discussed and a new total information measure is introduced. Various properties of the quantities in question are assessed by a simulation study. Heterogeneity test and measures are not directly related to the amount of between-study variance but to the relative increase of variance due to heterogeneity. It is more favourable to base the within-study variance estimate on the squared weights of individual studies than on the sum of weights. A heterogeneity measure scaled to a fixed interval needs reference values for proper interpretation. A measure defined by the relation of between- to within-study variance has a more natural interpretation but no upper limit. Both measures are quantifications of the impact of heterogeneity on the meta-analysis result as both depend on the variance of the individual study effects and thus on the number of patients in the studies.

Meta-analysis of genetic association studies: methodologies, between-study heterogeneity and winner's curse

“…In practice, max{0, I 2 } and maxf0; H 2 M g are used to restrict the ranges of these measures as positive. As the simulation study of Mittlbock and Heinzl, 59 unrestricted values of I 2 and H 2 M were used to obtain unbiased distributions for these measures in this study. These two measures presented monotonic increases as t 2 increased.…”

Endurance training for cancer patients during and shortly after completion of the anti-cancer treatment

“…Reporting of these items may reduce the level of bias in randomized clinical trials. Furthermore, bias between randomized controlled studies in metaanalyses may also be caused by heterogeneity of the research designs of the studies, such as different therapeutic dosages or exercise strategies [27].…”

MetaEasy: A Meta-Analysis Add-In for MicrosoftExcel

“…I 2 is deemed to be more reliable in assessing inconsistency between studies, with values of 25%, 50% and 75% corresponding to low, moderate and high heterogeneity respectively (Higgins, Thompson, Deeks, and Altman 2003). only measure that is truly independent of k and it takes values in the [0, +∞) range with 0 indicating -a somewhat worrying -perfect homogeneity (Mittlbock and Heinzl 2006). It has also been suggested that the estimate of between-study variance,τ 2 , should be reported (Higgins and Green 2008), therefore the estimated between-study variance, where relevant to the model, is also displayed.…”