Assessment of generalisability in trials of health interventions: suggested framework and systematic review

Abstract

Most evaluations of new treatments use highly selected populations, making it difficult to decide whether they would work elsewhere. Systematic evaluation and reporting of applicability is required Randomised trials of health interventions generally describe outcomes among participants with little consideration of whether the effects can be generalised. However, generalisability cannot be assumed with either biomedical interventions or more complex social interventions.w1 If their results are to be translatable into policy and practice decisions, trials must provide evidence about how relevant the interventions might be to other sites and populations.1 w2 Such information is particularly crucial for resource poor settings. 2Although CONSORT criteria for reporting randomised trials include assessment of generalisability, 3 a framework for empirically assessing and reporting this is lacking. We consider the factors affecting generalisability using examples from HIV and sexual health, examine how a sample of trials looked at generalisability, and suggest how to improve evaluation.

Effectiveness of pharmacist intervention for asthma control improvement

“…The brief, simple and pragmatic intervention carried out in the present study is feasible to implement in all community pharmacies, which argues for the generalisability of the present results. Moreover, the random selection of the pharmacies participating in the present study, and the community-based patient selection using fairly straightforward inclusion criteria also pleads for the generalisability of the study results [35].…”

Transporting Evidence-Based Parenting Programs for Child Problem Behavior (Age 3–10) Between Countries: Systematic Review and Meta-Analysis

“…Although the theoretical basis and process for translating effective interventions according to cultural or national differences is not fully understood, there is a growing literature on this topic. For example, models have been developed for examining applicability of public health evidence across contexts (e.g., Bonell, Oakley, Hargreaves, Strange, & Rees, 2006;Burchett, Umoquit, & Dobrow, 2011;Wang, Moss, & Hiller, 2006), and frameworks, derived from mental health and prevention literature, to guide translation and adaptation (Barrera & Castro, 2006;Ferrer-Wreder, Sundell, & Mansoory, 2012;Kumpfer, Pinyuchon, de Melo, & Whiteside, 2008;Sussman, Unger, & Palinkas, 2008). Moreover, there is a growing body of literature-though no systematic review-on factors affecting whether transportation is likely to be appropriate or successful.…”

External Validity of Knee Osteoarthritis Clinical Trials: A Systematic Review

“…24 For this reason, it was argued that the CONSORT statement lacked the reporting of external validity. 25 Alternative frameworks were proposed, such as qualitative studies or quality assessment checklists. 25,26 In the meantime, one method that aims to gauge external validity quantitatively was proposed.…”

“…25 Alternative frameworks were proposed, such as qualitative studies or quality assessment checklists. 25,26 In the meantime, one method that aims to gauge external validity quantitatively was proposed. 8 Specifically, an outcome of NNS was magnified.…”