How can systematic reviews incorporate qualitative research? A critical perspective

Abstract

Systematic review has developed as a specific methodology for searching for, appraising and synthesizing findings of primary studies, and has rapidly become a cornerstone of the evidence-based practice and policy movement. Qualitative research has traditionally been excluded from systematic reviews, and much effort is now being invested in resolving the daunting methodological and epistemological challenges associated with trying to move towards more inclusive forms of review. We describe our experiences, as a very diverse multidisciplinary group, in attempting to incorporate qualitative research in a systematic review of support for breastfeeding. We show how every stage of the review process, from asking the review question through to searching for and sampling the evidence, appraising the evidence and producing a synthesis, provoked profound questions about whether a review that includes qualitative research can remain consistent with the frame offered by current systematic review methodology. We conclude that more debate and dialogue between the different communities that wish to develop review methodology is needed, and that attempts to impose dominant views about the appropriate means of conducting reviews of qualitative research should be resisted so that innovation can be fostered.

Influence of organisational culture on the implementation of health sector reforms in low- and middle-income countries: a qualitative interpretive review

“…Although founded on the principles of systematic review [24,25], such an approach goes beyond a review of literature to generate new concepts and meanings from synthesis of the collated work [25,26]. These new ideas are, in essence, analytic generalisations of potential relevance in settings beyond those specifically considered in the papers reviewed [27].…”

Which standards from which disciplines? A test of systematic review for designing interdisciplinary evaluations

“…There is debate over the value of external quality criteria in assessing qualitative studies. While SRs traditionally excluded outright poor quality studies, the view has been expressed that poor quality studies might still provide valuable insight in qualitative reviews, leading some to apply quality screening to reviews but to nonetheless include those articles judged to be of poor quality -for a discussion on the value of quality criteria, see Campbell et al (2003), Dixon-Woods et al (2006a) and Edwards, Russell, and Stott (1998) and for reviews which include negatively assessed studies, see Dixon-Woods et al (2006b), Lemmer, Grellier, and Steven (1999) and McPherson and Armstrong (2012). We approached this review with the realist intention of generating prescriptions for research that would 'hold water' in the real world.…”

“…interventions through randomised controlled trials (RCTs) and subsequent meta-analysis of RCTs through SR (Dixon-Woods et al 2006a;Evans and Benefield 2001;Magarey 2001;Major and SavinBaden. 2012).…”

Systematic reviews: from ‘bare bones’ reviews to policy relevance

“…5. Spencer et al (2003) and Dixon-Woods et al (2006) Noyes et al (2011) and Saini and Shlonsky (2012) for useful overviews. 7.…”

“…There are a wide range of perspectives, from a total rejection of the application of quality criteria to qualitative research, to those advocating applying common concepts to both quantitative and qualitative research (Spencer et al 2003). Already 6 years ago, over 100 different checklists, quality criteria and standards for appraising the quality of qualitative research had been developed (Dixon-Woods et al 2006).…”