Theories of Change and Realistic Evaluation

Blamey, Avril; Mackenzie, Mhairi

doi:10.1177/1356389007082129

Cited by 298 publications

(160 citation statements)

References 23 publications

Supporting

Mentioning

135

Contrasting

Unclassified

Order By: Relevance

“…However the authors are of the view that the identity of so-called "generative mechanism" is the essence of programme theory; thus a theory of change itself is not the same as "M", the mechanism. A similar argument is developed by Blamey and Mackenzie (2007) hypotheses. A set of identified CMO configurations for efficiency criterion is shown in Table 13.4.…”

Section: Context For Efficiencysupporting

confidence: 63%

What Do Evaluations Tell Us About Climate Change Adaptation? Meta-analysis with a Realist Approach

Miyaguchi

Uitto

2017

Evaluating Climate Change Action for Sustainable Development

View full text Add to dashboard Cite

Evaluating climate change adaptation (CCA) interventions has yet proved to be a difficult task, as they involve a number of different stakeholders, time and geographical scale and political jurisdictions. As one effort to shed light on the subject, this paper presents the methodology and the results of a metaanalysis of ex-post evaluations of CCA programmes using a realist approach. This paper analyses CCA programmes in nine countries: Armenia, Egypt, Malawi, Mozambique, Namibia, the Philippines, Tanzania, Turkey and Zimbabwe. Together with their respective host governments, these programmes were implemented by either UNDP or various United Nations partner agencies and have already been evaluated by independent evaluators. Based on the analytical frameworks for evaluating CCA interventions, the authors hypothesized a number of key context, mechanism, and outcome configurations, which are considered vital in realist evaluation approach but have not yet been widely tested in the field of CCA. Although ex-post evaluations of multi-donor funded projects tend to be prepared out of bureaucratic requirement, the analytical method used in this paper, if used carefully, can unearth otherwise hidden important lessons and provide useful explanations. The results of the analysis can indicate that adopting a realist approach to complex development projects, such as these CCA programmes, is indeed a useful way of providing applicable explanations, rather than judgments, of what types of interventions may work for whom, how and in what circumstances for future CCA programming.

show abstract

Section: Context For Efficiencysupporting

confidence: 63%

What Do Evaluations Tell Us About Climate Change Adaptation? Meta-analysis with a Realist Approach

Miyaguchi

Uitto

2017

Evaluating Climate Change Action for Sustainable Development

View full text Add to dashboard Cite

show abstract

“…9 Logic models have been suggested as a means to help to provide a strategic perspective on complex programmes and to understand the relationships between various elements of an intervention and outcomes. 10 In particular, they are recommended for evaluating highly complex, multisite interventions with multiple and/or indeterminate outcomes. 11 The area of referral/demand management has many of the same challenges as other complex interventions.…”

Section: Logic Modelsmentioning

confidence: 99%

What is the evidence on interventions to manage referral from primary to specialist non-emergency care? A systematic review and logic model synthesis

Blank

Baxter

Woods

et al. 2015

Health Services and Delivery Research

View full text Add to dashboard Cite

BackgroundDemand management describes any method used to monitor, direct or regulate patient referrals. Several strategies have been developed to manage the referral of patients to secondary care, with interventions targeting primary care, specialist services, or infrastructure.ObjectiveThis research aimed to conduct an inclusive systematic review and logic model synthesis in order to better understand factors impacting on the effectiveness of interventions targeting referral between primary and secondary medical health care.DesignThe approach combined systematic review with logic modelling synthesis techniques to develop an evidence-based framework of factors influencing the pathway between interventions and system-wide changes.SettingPrimary health care.Main outcome measuresReferral from primary to secondary care.Review methodsSystematic searches were undertaken to identify recent, relevant studies. Quality of individual studies was appraised, with consideration of overall strength of evidence. A narrative synthesis and logic model summary of the data was completed.ResultsFrom a database of 8327 unique papers, 290 were included in the review. The intervention studies were grouped into four categories of education interventions (n = 50); process change interventions (n = 49); system change interventions (n = 38); and patient-focused interventions (n = 3). Effectiveness was assessed variously in these papers; however, there was a gap regarding the mechanisms whereby these interventions lead to demand management impacts. The findings suggest that, although individual-level interventions may be popular, the stronger evidence relates only to peer-review and feedback interventions. Process change interventions appeared to be more effective when the change resulted in the specialist being provided with more or better quality information about the patient. System changes including the community provision of specialist services by general practitioners, outreach provision by specialists and the return of inappropriate referrals appeared to have evidence of effect. The pathway whereby interventions might lead to service-wide impact was complex, with multiple factors potentially acting as barriers or facilitators to the change process. Factors related, first, to the doctor (including knowledge, attitudes and beliefs, and previous experiences of a service), second, to the patient (including condition and social factors) and, third, to the influence of the doctor–patient relationship. We also identified a number of potentially influential factors at a local level, such as perceived waiting times and the availability of a specialist. These elements are key factors in the pathway between an intervention and intended demand management outcomes influencing both applicability and effectiveness.ConclusionsThe findings highlight the complexity of the referral process and multiple elements that will impact on intervention outcomes and applicability to a local area. Any interventions seeking to change referral practice need to address factors relating to the individual practitioner, the patient and also the situation in which the referral is taking place. These conclusions apply especially to referral management in a UK context where this whole range of factors/issues lies well within the remit of the NHS. This work highlights that intermediate outcomes are important in the referral pathway. It is recommended that researchers include measure of these intermediate outcomes in their evaluation of intervention effectiveness in order to determine where blocks to or facilitators of system-wide impact may be occurring.Study registrationThe study is registered as PROSPERO CRD42013004037.FundingThe National Institute for Health Research Health Services and Delivery Research programme.

show abstract

“…a policy or initiative -and may include political, organisational, economic and social features (Blamey & Mackenzie, 2007). It is now widely accepted that context matters when evaluating initiatives in order to generate policy relevant evidence for their effects on inequalities in health (Poland, Frohlich, & Cargo, 2008;Shoveller et al, 2015).…”

Section: Introductionmentioning

confidence: 99%

Putting context centre stage: evidence from a systems evaluation of an area based empowerment initiative in England

Orton

Halliday

Collins

et al. 2016

Critical Public Health

View full text Add to dashboard Cite

Use policyThe full-text may be used and/or reproduced, and given to third parties in any format or medium, without prior permission or charge, for personal research or study, educational, or not-for-prot purposes provided that:• a full bibliographic reference is made to the original source • a link is made to the metadata record in DRO • the full-text is not changed in any way The full-text must not be sold in any format or medium without the formal permission of the copyright holders.Please consult the full DRO policy for further details. ABSTRACTIt is now widely accepted that context matters in evaluations of the health inequalities impact of community-based social initiatives. Systems thinking offers a lens for uncovering the dynamic relationship between such initiatives and their social contexts. However, there are very few examples that show how a systems approach can be applied in practice and what kinds of evidence are produced when this happens. In this paper, we use data from ethnographic fieldwork embedded within a multi-site mixed method evaluation to demonstrate how a systems approach can be applied in practice to evaluate the early stages of an area-based empowerment initiative -Big Local (funded by the Big Lottery Fund and delivered by Local Trust). Taking place in 150 different local areas in England and underpinned by an ethos of resident-led collective action, Big Local offers an illustration of the applicability of a systems approach to better understand the change processes that emerge as social initiatives embed and co-evolve within a series of local contexts. Findings reveal which parts of the social system are likely to be changed, by what mechanisms, and with what implications. They also raise some salient considerations for knowledge generation and methods development in public health evaluation, particularly for the evaluation of social initiatives where change does not necessarily happen in linear or predictable ways. We suggest future evaluations of such initiatives require the use of more flexible designs, encompassing qualitative approaches capable of capturing the complexity of relational systems processes, alongside more traditional quantitative methods.

show abstract

Theories of Change and Realistic Evaluation

Cited by 298 publications

References 23 publications

What Do Evaluations Tell Us About Climate Change Adaptation? Meta-analysis with a Realist Approach

What Do Evaluations Tell Us About Climate Change Adaptation? Meta-analysis with a Realist Approach

What is the evidence on interventions to manage referral from primary to specialist non-emergency care? A systematic review and logic model synthesis

Putting context centre stage: evidence from a systems evaluation of an area based empowerment initiative in England

Contact Info

Product

Resources

About