Applying and reporting relevance, richness and rigour in realist evidence appraisals: Advancing key concepts in realist reviews

Abstract

The realist review/synthesis has become an increasingly prominent methodological approach to evidence synthesis that can inform policy and practice. While there are publication standards and guidelines for the conduct of realist reviews, published reviews often provide minimal detail regarding how they have conducted some methodological steps. This includes selecting and appraising evidence sources, which are often considered for their ‘relevance, richness and rigour.’ In contrast to other review approaches, for example, narrative reviews and meta‐analyses, the inclusion criteria and appraisal of evidence within realist reviews depend less on the study's methodological quality and more on its contribution to our understanding of generative causation, uncovered through the process of retroductive theorising. This research brief aims to discuss the current challenges and practices for appraising documents' relevance, richness and rigour and to provide pragmatic suggestions for how realist reviewers can put this into practice.

Shared decision-making interventions in the choice of antipsychotic prescription in people living with psychosis (SHAPE): protocol for a realist review

“…For those articles deemed to meet the inclusion criteria, IF will retrieve the full text and classify studies into high and low relevance, depending on their relevance to the programme theory, and based on established methods previously employed. [32,43,44] A random subsample of the 10% of final documents for inclusion will be selected and assessed independently by another member of the research team (LS/EC/IM/JH/EW) to identify systematic errors. [26, 33] The remaining 90% of decisions will be made by IF, although a number of these may require further discussion/joint reading between the wider project team due to issues of uncertainty regarding relevance and/or rigour.…”

A Rapid Realist Review on Leadership and Career Advancement Interventions for Women in Healthcare

“…Studies that solely focused on the reporting of barriers or enablers related to gender equity broadly, with no interventional element or that did not include at least one outcome related to advancing women in leadership or their careers Evaluating Richness and Rigour Papers and documents deemed relevant to the research topic were reviewed for richness. Richness assessments were based on the inclusion of su cient depth to meaningfully contribute to theory building as indicated by having traceable CMOcs; theories of interest were either the initial programme theories or other theories relevant to the topic under scrutiny [22]. Studies were scored 1 point for each CMOc that was evident.…”

“…Rigour was applied to assess the methodological conduct of the included papers and documents. Rigour was assessed based on a yes (1) or no (0) dichotomy for the credibility of the source, appropriateness and trustworthiness of methodology used, and plausibility of the information reported [22]. Summary of richness and rigour scores can be found in Appendix 2.…”

Understanding strategies that foster nurses to act as clinical leaders in hospitals: A realist review

“…Realist reviews seek to explain complex interventions by synthesizing evidence from varied designs (Pawson et al., 2005). That is why realist reviews appraise the relevance and rigour of sources instead of using conventional approaches for quality appraisal (Dada et al., 2023; Pawson et al., 2005). In line with the RAMESES standards, the sources were appraised on relevance , which refers to whether the source “can contribute to theory building and/or testing” and on rigour , which refers to screening ‘whether or not the method used to generate that particular piece of data is credible and trustworthy’ (Wong et al., 2014, p. 35).…”