Integrating theory-based evaluation and process tracing in the evaluation of civil society gender budget initiatives

Abstract

Over the last two decades, gender-responsive budgeting has gained prominence as an effective tool for governments to fulfil gender commitments and the realisation of women's rights. To date, however, limited empirical evidence exists of the impact and effectiveness of gender budget initiatives. This article proposes and demonstrates the integration of theory-based evaluation and process tracing to examine the effects of local-level civil society-led gender-responsive budgeting on maternal health service delivery in Kabale District, rural Uganda. It involves four steps: explicating the programme theory linking the gender budget initiative to the intended outcome; theorising the underlying causal mechanism; making case-specific predictions of observable manifestations of the mechanism; and testing the empirical evidence using Bayesian logic to make causal inferences about the effects of the civil society gender budget initiative on maternal health service delivery. This approach strengthens our confidence in the inferences made about causality and the effects of gender budget initiatives.

Multi-Method Research in the Social Sciences: A Review of Recent Frameworks and a Way Forward

“…Finally, Gerring’s claim about policy relevance does not match recent developments in the field of policy evaluation, where there is increasing interest in the tracing of mechanisms as an analytical tool to study how interventions work in particular contexts instead of working with broad propositions that tell us little about how things work in the real world (Bamanyaki and Holvoet 2016; Cartwright 2011; Cartwright and Hardie 2012; Clarke et al 2014; Schmitt and Beach 2015; Wauters and Beach 2018).…”

Process tracing and congruence analysis to support theory-based impact evaluation

“…Schmitt and Beach (2015) develop such a chain for a part of the reasoning behind budget support in the context of development aid. Another example, concerning Gender Responsive Budgeting, is provided by Bamanyaki and Holvoet (2016).…”

Making rigorous causal claims in a real-life context: Has research contributed to sustainable forest management?

“…Step two consists in predicting some facts that should be observed for each part of the theory to pass empirical tests, which are the third and last step (Bamanyaki and Holvoet, 2016;Beach and Pedersen, 2013;Rohlfing, 2014).…”

“…This dimension echoes the Process Tracing (PT) method, which was originally developed for investigating historical events (Alexander and Bennett, 1997; Falleti, 2006). The method was further refined in terms of epistemology (Beach and Pedersen, 2013; Bennett and George, 1997), transferred to the world of evaluation (Bamanyaki and Holvoet, 2016; Befani, 2012; Schmitt and Beach, 2015), and even connected with CA (Befani and Mayne, 2014). In its present state, PT provides a valuable framework for understanding the underlying logic of our contribution claims.…”

“…In step one, the evaluator refines the theory of change into clearly elicited hypotheses on all causes (Xs), effects (Ys), processes through which Xs generate Ys, and conditions under which these processes operate correctly. Step two consists in predicting some facts that should be observed for each part of the theory to pass empirical tests, which are the third and last step (Bamanyaki and Holvoet, 2016; Beach and Pedersen, 2013; Rohlfing, 2014).…”