Panning for the gold in health research: Incorporating studies’ methodological quality in meta-analysis

Abstract

In large literatures exhibiting heterogeneous effects, meta-analyses can incorporate methodological quality and generate conclusions that enable greater confidence not only about the substantive phenomenon but also about the role that methodological quality itself plays.

Assessing the Existing Professional Exercise Recommendations for Hypertension: A Review and Recommendations for Future Research Priorities

“…3 Following fundamental scientific methods, higher-quality interventions suggest lower bias and higher validity. 10,11 Yet, this "best evidence approach" 12 often limits included studies to RCTs exclusively, and despite however strong their methodological quality, RCTs are but one dimension of the literature. 10,[13][14][15] Scientific discovery and its translation to clinical practice rely on the replication of findings from independent groups to ensure the robustness of the observed effects across relevant parameters.…”

The question-behaviour effect: A theoretical and methodological review and meta-analysis

“…Simple slopes analyses (see Johnson, Low, & MacDonald, 2014) indicated that social desirability was associated with effect sizes when dissonance was low (β = 0.211, 95% CI = 0.10, 0.32, p < .001) but not when dissonance was high (β = 0.079, 95% CI = −0.003, 0.16, p = .06). Thus, the social desirability of the behaviour appears to be an important guide to acting on self-predictions/intentions when inaction arouses little dissonance.…”

Risk of bias in randomised controlled trials of health behaviour change interventions: Evidence, practices and challenges

“…Guyatt et al, 2011;Higgins, Altman, & Sterne, 2011). These tools are also influential: trials scoring high on risk-of-bias assessments should have a smaller chance on publication and -if published -to be included in 'best evidence' systematic reviews (Johnson, Low, & MacDonald, 2015), and thus for their interventions to influence policy and practice. Systematic reviews suggest that many health behaviour change (HBC) trials suffer from a moderate to high risk of bias (e.g.…”

“…Last but not least, Johnson et al (2015) focus on the question how meta-analyses of health promotion interventions incorporate methodological quality (including the risk of bias) in their analyses. Based on a survey of 200 meta-analyses, they arrive at the conclusion that many do assess methodological quality, but very few incorporate them in their analyses.…”