Beyond synthesis: augmenting systematic review procedures with practical principles to optimise impact and uptake in educational policy and practice

Abstract

Whilst systematic reviews, meta-analyses and other forms of synthesis are often constructed as sitting proudly atop the hierarchy of research evidence, their limited impact on educational policy and practice has been criticised. In this article, we analyse why systematic reviews do not benefit users of evidence more consistently and suggest how review teams can optimise the impact of their work. We introduce the Beyond Synthesis Impact Chain (BSIC), an integrated framework of practical strategies for enhancing the impact of systematic reviews. Focusing upon examples from health professions education, we propose that review teams can optimise the impact of their work by employing strategies that 1) focus on practical problems and mindful planning in collaboration with users; 2) ensure reviews are relevant and syntheses reflexively account for users' needs; and 3) couch reports in terms that resonate with users' needs and increase access through targeted and strategic dissemination. We argue that combining practical principles with robust and transparent procedures can purposefully account for impact, and foster the uptake of review evidence in educational policy and practice. For systematic review teams, this paper offers strategies for enhancing the practical utility and potential impact of systematic reviews and other forms of synthesis.Keywords: systematic review, impact, knowledge synthesis, evidence-based practice 2 IntroductionThe rhetoric of evidence-based practice is ubiquitous. Since the late 1990s, practitioners and policymakers across a range of fields have been expected to ensure that their practices and policies are underpinned with rigorous research that robustly demonstrates 'what works' (Wells, 2007). As a result mechanisms and procedures for locating, assessing the quality of, and synthesising evidence were, and continue to be, constructed. Systematically synthesised evidence, in the form of the systematic review, is often favoured by policymakers in education and the social sciences (Rubin and Bellamy, 2012; Solesbury, 2001). The transparent and auditable procedures of systematic reviews, along with their rigorous assessments of methodological quality, are lauded as providing conclusions that far exceed the validity of individual studies alone. In short, the message of the rhetoric is 'to base your practices and policies on evidence, locate or commission a systematic review that responds to your specific practical problem.' Systematic reviews were pioneered internationally in the medical arena by the Cochrane Collaboration, but have crossed over into social policy with the establishment of the Campbell Collaboration, and in the United Kingdom the ESRC UK Centre for EvidenceBased Policy and Practice (Solesbury, 2001), the Evidence for Policy and Practice Information (EPPI) Co-ordinating Centre, and the government's creation of the What Works Network to support public services (Cabinet Office, 2014). In the field of health professions education, the Best Evidence Medical and Health Profess...

Understanding academics' adoption of learning technologies: A systematic review

“…We are mindful of the need to address the interests of potential users of systematic reviews (Green, Taylor, Buckley, & Hean, 2016). We imagine that the users of the review will be academic developers, champions and middle managers.…”

What is the evidence on the best way to get evidence into use in education?

“…Even the best systematic syntheses of evidence often have little impact in practice, without further translation and activity (Haines et al , 2004; Green et al , 2016). So how should research findings be converted into more usable forms?…”

Is the educational ‘what works’ agenda working? Critical methodological developments

“…The first (Higgins and Katispataki 2016) and the last two papers (Anwaruddin 2016;Green et al 2016) are also related to dissemination of findings with the relevant stakeholders addressing directly the impact agenda of educational research which is becoming prominent in the various funding agendas (e.g. Economic and Social Research Council, ESRC) as well as academic/research evaluation-reformed agendas (e.g.…”

“…This issue also covers a range of methodological approaches, including systematic reviews and meta-analysis (Green et al 2016;Higgins and Katsipataki 2016), advancements of methods for dealing with the complexity involved in interventions by either improved models (Schweig and Pane 2016;Spybrook, Shi, and Kelcey 2016) and/or integrated approaches (Hanley, Cambers, and Haslam 2016;Peterson 2016).…”

“…The topics covered ranged from language teaching (Anwaruddin 2016), health profession education (Green et al 2016), science and maths teaching (Hanley, Cambers, and Haslam 2016) and voluntary summer learning programmes (Schweig and Pane 2016).…”