Meeting the challenge: developing systematic reviewing in social policy

Abstract

English This article uses the experience of reviewing the evidence on the financial support available for defaulting home owners to consider the opportunities and challenges systematic review methods present to social policy. It addresses concerns about examining the strength of given evidence, and perceptions of it being a purely technical method to review existing research. It argues that there is merit in utilising the method to provide research users with transparent summaries of the most robust evidence with minimum bias. The article outlines the challenges presented and suggests that social policy researchers have a valuable contribution to make to the developing methods.

Reviewing the evidence: reflections from experience

“…In theory, the systematic review process of "locating, appraising, and synthesising evidence" (Petticrew, 2001) seems a simple and promising -if arduoustask, but numerous difficulties emerge while trying to conduct it (Wallace et al, 2004). …”

“…The majority of publications are theoretical and concerned with how EBP research could be practised; they do not discuss the methodological challenges and practical problems that arise from its actual conduct (Klein, 2000;Pawson, 2001aPawson, , 2001bPetticrew, 2001;Boaz et al, 2002;Packwood, 2002;Sanderson, 2002;Solesbury, 2002;Young et al, 2002;Wallace et al, 2004). This has given rise to the impression that such research is straightforward and uncomplicated, requiring only the technical application of the methodology in order to get the desired results.…”

Which standards from which disciplines? A test of systematic review for designing interdisciplinary evaluations

“…In the social sciences, SRs are now also used to make sense of and manage the 'information explosion', separate wheat from the chaff, identify gaps in an evidence base, confirm, refute, develop or modify bodies of theory, and increase the standard of research in the field (Bondas and Hall 2007;Campbell et al 2003;Langer and Stewart 2014;Major and SavinBaden. 2012;Noblit and Hare 1988;Wallace et al 2004;White and Waddington 2012). SRs are now used in fields such as education and training (Evans and Benefield 2001;Price 2005;Secomb 2008), social policy (Wallace et al 2004) and experiential research in health care (Arman and Rehnsfeldt 2003;Campbell et al 2003;Dixon-Woods et al 2007, 2006bHughes, José Closs, and Clark 2009).…”

“…2012;Noblit and Hare 1988;Wallace et al 2004;White and Waddington 2012). SRs are now used in fields such as education and training (Evans and Benefield 2001;Price 2005;Secomb 2008), social policy (Wallace et al 2004) and experiential research in health care (Arman and Rehnsfeldt 2003;Campbell et al 2003;Dixon-Woods et al 2007, 2006bHughes, José Closs, and Clark 2009). The expanding range of domains in which SR is applied and its suitability for working from a broad literature commended it to our use.…”

“…11. For an example of a strategy of using quality to accord significance, see NHS CRD (2001; cited in Wallace et al 2004). 12. Notable exceptions include Clegg (2005) who cites communications from research centres, quality standards bodies and a medical institute.…”