A Contribution to Current Debates in Impact Evaluation

Abstract

A debate on approaches to impact evaluation has raged in development circles in recent years. This paper makes a contribution to this debate through discussion of four issues. First, I point out that there are two definitions of impact evaluation. Neither is right or wrong, but they refer to completely different things. There is no point in methodological debates unless they agree a common starting point. Second, I argue that there is confusion between counterfactuals, which are implied by the definition of impact evaluation adopted in this paper, and control groups, which are not always necessary to construct a counterfactual. Third, I address contribution rather than attribution — a distinction that is also definitional, mistaking claims of attribution to mean sole attribution. I then consider accusations of being ‘positivist’ and ‘linear’, which are, respectively, correct and unclear. Finally, I suggest that these arguments do not mean that there is a hierarchy of methods, rather quantitative approaches, including RCTs, are often the most appropriate methods for evaluating the impact of a large range of interventions, having the added advantage of allowing analysis of cost effectiveness or cost-benefit analysis.

Livelihood implications of in situ-on farm conservation strategies of fruit species in Uzbekistan

“…One of the most common objective faced by institutions and scholars that implement development programs aiming to promote the sustainable use of agricultural diversity in rural settings is the identification of the intervention pathways needed to reach a significant and tangible impact. The assessment presented in this study considers the sustainability of the impact (White 2010) as a key aspect of the development efficacy. From an empirical point of view, the assessment was carried out using a quasi-experimental (observational) design for capturing the impact of project participation on target fruit tree diversity, self-consumption, marketing and household livelihoods.…”

“…Despite the short-term outcomes of external interventions in areas such as genetic diversity conservation, wherein yields and agronomic management are well-documented, there is a lack of structured and analytic assessment of the success of these projects in broader and longer gains beyond immediate training-development efficacy- (Lutz and Munasingheb 1994), while the majority of existing impact studies fail to recognize that a key aspect of the development efficacy is the sustainability of the impact (White 2010). The latter is pivotal for assessing the long-term efficacy of activities and their real impact on rural livelihoods (Bellon et al 2015a).…”

“…The latter is pivotal for assessing the long-term efficacy of activities and their real impact on rural livelihoods (Bellon et al 2015a). Tracing impact over time presents problems that require rethinking or adapting existing methodologies or creating new impact assessment approaches across temporal scales (White 2010).…”

The Role of Blended Value Accounting in the Evaluation of Socio-Economic Impact of Social Enterprises

“…The ACA determines how outcomes would have changed if the intervention had not been undertaken (White 2010). This involves counterfactual analysis, that is, ''a comparison between what actually happened and what would have happened in the absence of the intervention'' (White 2009).…”

The growth of impact evaluation for international development: how much have we learned?

“…By 'impact evaluation', we refer to counterfactual-based programme evaluation that attempts to attribute specific outcomes to programmatic activities by dealing with the problem of selection bias (for a complete discussion, see White 2010). As the evaluation gap working group argued in its 2006 report 'Will we ever learn?…”