Evidence-based practitioners who want to apply evidence from complex interventions to the care of their patients are often challenged by the difficulty of grading the quality of this evidence. Using the GRADE (Grading of Recommendations, Assessment, Development and Evaluation) approach and an illustrative example, we propose a framework for evaluating the quality of evidence that depends on obtaining feedback from the evidence user (eg, guideline panel) to inform: (1) proper framing of the question, (2) judgements about directness and consistency of evidence and (3) the need for additional contextual and qualitative evidence. Using this framework, different evidence users and based on their needs would consider the same evidence as high, moderate, low or very low.Complex interventions in medicine are defined as interventions that contain several interacting components within the experimental and control interventions, multiple behaviours required by those delivering or receiving the intervention, multiple groups or organisational levels targeted by the intervention, and permit for variable flexibility and tailoring of the intervention.1 An example is care management programmes for type 2 diabetes. In one systematic review, programmes varied across studies in terms of team composition ( physician, nurse, case manager, diabetes educator), delivery method (face-to-face, telephone, online, outpatient, inpatient), intensity and frequency of the intervention, and other factors. 2 Complex interventions are increasingly used in studies of behavioural change, psychotherapy, education, public health, health services, quality improvement, social policy and many other fields.
The challengeJust as with any other intervention, evidence-based practitioners who want to apply evidence from complex intervention to the care of their patients need to select the evidence (hopefully through a systematic review process) and appraise it (ie, identify the extent to which this evidence is trustworthy, also called, rating the quality of evidence). The GRADE (Grading of Recommendations, Assessment, Development and Evaluation) approach is a modern method for rating the quality of evidence with good transparency and reliability. 3 A study of Cochrane systematic reviews showed that the outcomes of complex interventions were more likely to be rated as 'very low' quality of evidence compared with those of simple interventions (37.5% vs 9.1%). None of the outcomes of complex intervention reviews were rated as 'high'. 4 We believe these low ratings are inconsistent with the definition of quality of evidence in GRADE, which is a construct that reflects the trustworthiness of evidence, and we attribute this phenomenon to improper framing of the clinical question for which the quality of evidence is being rated.
5
RationaleComplex interventions are inherently heterogeneous; thus, we may be tempted to lower the quality of evidence for heterogeneity (inconsistency across studies). Complex interventions are also likely to have components that a...