The “realist search”: A systematic scoping review of current practice and reporting

Abstract

The requirement for literature searches that identify studies for inclusion in systematic reviews should be systematic, explicit, and reproducible extends, at least by implication, to other types of literature review. However, realist reviews commonly require literature searches that challenge systematic reporting; searches are iterative and involve multiple search strategies and approaches. Notwithstanding these challenges, reporting of the "realist search" can be structured to be transparent and to facilitate identification of innovative retrieval practices. Our six-component search framework consolidates and extends the structure advanced by Pawson, one of the originators of realist review: formulating the question, conducting the background search, searching for program theory, searching for empirical studies, searching to refine program theory and identifying relevant mid-range theory, and documenting and reporting the search process. This study reviews reports of search methods in 34 realist reviews published within the calendar year of 2016. Data from all eligible reviews were extracted from the search framework. Realist search reports poorly differentiate between the different search components. Review teams often conduct a single "big bang" multipurpose search to fulfill multiple functions within the review. However, it is acknowledged that realist searches are likely to be iterative and responsive to emergent data. Overall, the search for empirical studies appears most comprehensive in conduct and reporting detail. In contrast, searches to identify and refine program theory are poorly conducted, if at all, and poorly reported. Use of this framework offers greater transparency in conduct and reporting while preserving flexibility and methodological innovation. K E Y W O R D S literature searches, realist synthesis, reporting standards Highlights What is already knownRealist syntheses are becoming increasingly prevalent but methods for searching ("realist searches") are poorly specified. What is newRealist searches require iterative methods that use different search approaches to support different components of the realist synthesis process.This audit of descriptions of search components from published realist syntheses for a single calendar year reveals examples of consensus on candidate approaches for retrieval and reporting as well as instances of genuine innovation. Potential impact for RSM readers outside the authors' fieldAs realist syntheses start to populate subject fields where the systematic review convention has not previously gained traction it will be helpful for researchers in those fields to be exposed to information retrieval methods that can offer a systematic approach to study identification. This paper offers a framework for planning, reporting and evaluating future realist searches from across multiple subject fields in an ongoing quest to improve standards of conduct and reporting. BOOTH ET AL. 15 "Realist Search"a preferred label that describes all procedures used to identify doc...

The anatomy of an award-winning meta-analysis: Recommendations for authors, reviewers, and readers of meta-analytic reviews

“…Given this difficulty and that literature searches often occur in an iterative manner, where researchers are learning the parameters of the search as they conduct them (i.e., ''Realist Search''; Booth, Briscoe, & Wright, 2020), there is an incentive to filter or simplify the procedure and to not properly document such a fundamentally flawed process so as to not leave it open to critique from reviewers' potentially idealistic standards (Aguinis, Ramani, & Alabduljader, 2018). The result can be an implicit selection bias, where the body of articles is a subset of what is of interest (Lee, Bosco, Steel, & Uggerslev, 2017).…”

The Role of Trust and Hope in Antipsychotic Medication Reviews in Primary Care Settings: A Realist Review. 

“…Search terms included variations of terms for "antipsychotic medication" and "primary care". Initial scoping searches indicated a paucity of papers speci cally discussing antipsychotic medication management in primary care, so the search strategy for the main search was designed to maximise sensitivity, and reduce the risk of missing data related to any potential contexts or mechanisms ("Big Bang Approach"; (20). Search results and results from the initial scoping searches were screened for eligibility based on the criteria in Table 1.…”

Community mobilisation approaches to preventing and reducing adolescent multiple risk behaviour: a realist review protocol

“…Realist review methodology is a theory-driven, interpretive approach to evidence synthesis (38) developed by the work of Pawson et al (39-41). It has gained increasing popularity in addressing the challenge of ‘what works, for whom, under what circumstances and in what time period’ and is considered especially salient when data are complex, multi-layered and there is a need to understand complex relationships, interdependence, and mechanisms (42). A core component of realist reviews is to develop ‘middle-range realist programme theory’ (43) that explains how an intervention ‘works’ within what contexts.…”

“…Study characteristics will be extracted into a table to provide a descriptive overview of the types of community mobilisation interventions included, based on a ‘bespoke’ set of data extraction forms informed by the relevant literature (39). Realist reviews are structured through Context-Mechanisms-Outcome (CMO), comparable to PICO for traditional systematic reviews (42). Context, mechanisms and outcomes are extracted during the realist review and can be conceptualised as the ‘data’ that support evidence to support, reject or refine the programme theory (33).…”