Realist evaluation: an immanent critique

Abstract

This paper critically analyses realist evaluation, focussing on its primary analytical concepts: mechanisms, contexts, and outcomes. Noting that nursing investigators have had difficulty in operationalizing the concepts of mechanism and context, it is argued that their confusion is at least partially the result of ambiguities, inconsistencies, and contradictions in the realist evaluation model. Problematic issues include the adoption of empiricist and idealist positions, oscillation between determinism and voluntarism, subsumption of agency under structure, and categorical confusion between context and mechanism. In relation to outcomes, it is argued that realist evaluation's adoption of the fact/value distinction prevents it from taking into account the concerns of those affected by interventions. The aim of the paper is to use these immanent critiques of realist evaluation to construct an internally consistent realist approach to evaluation that is more amenable to being operationalized by nursing researchers.

Great Expectations: The Implementation of Integrated Care and Its Contribution to Improved Outcomes for People with Chronic Conditions

“…The importance of this interplay has been described most appropriately in the Context-Mechanisms-Outcomes (CMO) Model, which postulates that interventions only have successful outcomes when they introduce appropriate mechanisms in the appropriate social and cultural contexts [11]. However, there is no consensus on the definition and operationalisation of what exactly is meant by the concepts “context”, “mechanisms” and “outcomes” [1213]. This is problematic for the consistent application of the model to the collection and analysis of empirical data, as well as the comparison of findings across studies.…”

The possibility of critical realist randomised controlled trials

“…Pawson and Tilley’s definition of social mechanisms in general, and programme mechanisms in particular, as consisting of a combination of structure (‘the capacities derived from group membership’/‘social resources’) and agency (‘people’s choices’/‘participants’ reasoning’) stands in stark contrast to the critical realist insistence that agency and structure are different entities with different properties and powers [26]. …”

“…A clearer definition of those relationships is, therefore, required. To that end, we commend the model that conceives of observed outcomes as the result of the interaction of contextual mechanisms, programme mechanisms and human agency [26]. …”

“…This strategy can be used to build middle-range theories about processes of causation; model the intervention; refine theories to provide explanations concerning what works for whom in what circumstances; and inform those introducing efficacious interventions into open contexts of contextual mechanisms that may affect their sustained effectiveness [26]. …”

“…We contend that this qualitative stage should not confine itself to examination of the reasoning behind agents’ responses for the purpose of optimising the motivational appeal of the intervention. In addition to this instrumental function, it should also attempt to illuminate people’s experiences of the intervention for the purpose of critically establishing its beneficial or detrimental effects upon the lives of those affected by it [26]. …”