A Quality-Effects Model for Meta-Analysis

Abstract

We introduce a quality-effects approach that combines evidence from a series of trials comparing 2 interventions. This approach incorporates the heterogeneity of effects in the analysis of the overall interventional efficacy. However, unlike the random-effects model based on observed between-trial heterogeneity, we suggest adjustment based on measured methodological heterogeneity between studies. We propose a simple noniterative procedure for computing the combined effect size under this model and suggest that this could represent a more convincing alternative to the random effects model.

Comorbidities, Exposure to Medications, and the Risk of Community-Acquired Clostridium difficile Infection: A Systematic Review and Meta-analysis

“…Rather, the quality score is used to rank studies by methodologic rigor and this rank is then linked with a synthetic bias variance that is added to the random error variance. 20 The other model used was the IVhet model that does not require input of quality information and so is less rigorous than the QE model. 22 Both of the latter models use a quasi-likelihood-based variance structure without distributional assumptions and thus have coverage probabilities for the confidence interval (CI) well above the nominal level.…”

“…This was justified because some have expressed skepticism regarding the appropriateness of the conventional RE model 16 owing to its documented underestimation of the statistical error, which leads to overconfident results. 5,[17][18][19] The other 2 models that were used were the quality effects (QE) model [20][21] and a novel method, the inverse variance heterogeneity (IVhet) model. 22 The QE model uses the Qi to redistribute the inverse variance weights in favor of the studies with higher methodologic quality and thus studies that provided higher quality of evidence contributed with a higher weighting towards the overall effect size.…”

Evidence Synthesis for Medical Decision Making and the Appropriate Use of Quality Scores

“…These models have been criticized for the fictional assumption that there is a random sample of studies in a meta-analysis 19 or ignoring study traits such as differences in the way in which treatment protocols were designed or conducted. 6,20 So, rather than add a fictional residual random treatment effect to the model, this paper diverges from other authors 4,7 by assuming that while the q can be treated as fixed, the effects of bias on the uncertainty of estimates can be included by the creation of a synthetic standard error that includes a component of variance for between study bias, 21,22 and thus, model 1 becomes…”

“…4 In agreement with Greenland, there has been the emergence of component item quality modeling schemes, 15,16 but this paper will attempt to show that, in general, correct quality-score weighting methods can produce less variable effect estimates, as has been suggested by some authors who continue to use and even promote such scores. 6,17,18 This justification stands, even when the component quality items do not capture all traits contributing to bias in study-specific estimates. Furthermore, an attempt will be made to demonstrate if the possible decrease in variance with such quality weighting can be a worthwhile trade-off against bias and how it performs against the random effect methods.…”

“…3 If the quality items or traits are assigned a number of points based on the a priori judgment of clinical investigators, they can then be summed into a univariate "quality score" that has previously been given the interpretation of capturing the essential features of a "multidimensional quality space" 3,4 and has been used as a weighting factor in averaging across studies. [5][6][7] or as a covariate for stratifying study results. 8,9 Greenland has raised scientific objections to the creation of this sort of univariate quality score from quality items.…”