Process tracing and congruence analysis to support theory-based impact evaluation

Wauters, Bénédict; Beach, Derek

doi:10.1177/1356389018786081

Cited by 17 publications

(17 citation statements)

References 20 publications

(56 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although it has existed as a methodology in social sciences for some time, particularly history and political science, it is increasingly used in theory-based impact evaluation Wauters and Beach 2018). Process tracing is used to explore and test causal inferences by critically analysing the sequence of events that have unfolded.…”

Section: A Process Tracing Approach To Assess Contribution Collaborat...mentioning

confidence: 99%

Untitled

2022

IDS Bulletin

View full text Add to dashboard Cite

AcknowledgementsThis IDS Bulletin would never have seen the light without the moral support and analytical insights of the Centre for Development Impact (CDI) and the Partnerships Resource Centre (PrC). CDI is a community of practice that reflects on appropriate, informative, and learning-oriented impact evaluation designs. It is a joint initiative between the Institute of Development Studies, the consultancy company Itad Ltd, and the University of East Anglia. PrC is an international research and knowledge centre for public-private collaboration for sustainable and inclusive development, embedded within the Rotterdam School of Management of the Erasmus University, the Netherlands. It engages with cross-sector partnerships through action research and supports strategizing and evaluative reflection processes. Funder acknowledgementsThis IDS Bulletin was financially supported by the 2SCALE programme, funded by the Netherlands Ministry of Foreign Affairs.

show abstract

Section: A Process Tracing Approach To Assess Contribution Collaborat...mentioning

confidence: 99%

Untitled

2022

IDS Bulletin

View full text Add to dashboard Cite

show abstract

“…This pragmatist position has consequences in relation to some of the distinctions found in some parts of realist evaluation. Great advances have been made in the development of methodological approaches to detecting causality in small-N studies (De Souza, 2013;Fontaine, 2020;Harris et al, 2019;Wauters and Beach, 2018), but the justification for these studies has sometimes been undergirded by dichotomies which privilege one side of these dichotomies (such as qualitative over quantitative studies, process-based studies over variance-based studies and deterministic causality over probabilistic causality). By doing so, realist evaluation has focused on how interventions work at the expense of how well they work.…”

Section: A Mechanism-oriented Approach To Evaluation Using Withinintervention Variationsmentioning

confidence: 99%

How and how well do workplace assessments work? Using contextual variations in a theory-based evaluation with a largeN

2021

View full text Add to dashboard Cite

What can be said about effect of an intervention without a control group? The lack of evaluative evidence is a long-standing problem for regulatory policies against work-related health and safety risks. The European Union Occupational Safety and Health Framework has been in operation for three decades and covers more than 200 million workers, but the most recent evaluation was inconclusive about the benefits generated by this framework. A theory-based evaluation focusing on mechanisms in combination with a design capturing within-intervention variations offers a way forward. The idea is to measure the prevalence of most likely mechanisms and their correlation with outcomes. This approach is illustrated in a large- N evaluation of the use of workplace assessments in the public sector in Denmark. The strengths and weakness of the workplace assessment legislation are assessed. It is shown how findings based on the presented approach contribute to the public debate about workplace assessments.

show abstract

“…While the methodological debate on research designs based on case studies has recently paid more attention to its applications for ex post evaluation, to test whether or not an hypothesized mechanism is causally connected to a certain outcome (Chen and Henry, 2020; Busetti and Dente, 2017), it can also be applied to ex ante evaluations aimed at assessing the coherence between policy design and goals (Melloni et al, 2016) and the conditions for evaluability (Wauters and Beach, 2018: 289). Different from process tracing, congruence analysis aims at assessing which among several theories better supports certain evidence (Blatter and Haverland, 2012; Wauters and Beach, 2018: 297) and thus is particularly useful to evaluate policy maker’s search for contextually valid solutions when they can choose among theories which are equally valid ‘on average’ (Bennett, 2015), with limited or no opportunities to learn from experience. The study of the two programmes used unstructured interviews as the main tool to relate the mechanisms operating in the specific context of intervention to the policy makers’ causal assumptions (Pawson, 1996).…”

Section: Two Policy Programmes Designed From Scratch: An Overview Of the Italian Casementioning

confidence: 99%

Preventing children’s death by heatstroke: How alternative accident theories affect policy design and evaluation

Giulio

2021

Evaluation

View full text Add to dashboard Cite

When public policies are designed from scratch, with no or limited possibility of learning from past experience, decision makers rely on their creativity and lateral thinking to counterbalance cognitive biases. One of the crucial issues for both evaluators and policy makers in such situations is not only to elaborate a sound theory of the programme but also to be aware of the existence of competing theories to solve the same problem. Being able to detect such theories constitutes an essential step to avoid ‘Type III errors’, that is, enacting the right solution for the ‘wrong’ problem. Reflecting on the analysis of two programmes aimed at preventing the death of children by heatstroke, this article illustrates the implications of alternative ‘accident theories’ for the evaluation of safety policies.

show abstract

Process tracing and congruence analysis to support theory-based impact evaluation

Abstract: Process tracing and congruence analysis to support theory based impact evaluation.

Cited by 17 publications

References 20 publications

Untitled

Untitled

How and how well do workplace assessments work? Using contextual variations in a theory-based evaluation with a largeN

Preventing children’s death by heatstroke: How alternative accident theories affect policy design and evaluation

Contact Info

Product

Resources

About