Exploring the Necessary Conditions for Evaluation Use in Program Change

Abstract

Research has identified a wide range of factors that affect evaluation use but continues to be inconclusive as to their relative importance. This article addresses the complex phenomenon of evaluation use in three ways: first, it draws on recent conceptual developments to delimitate the examined form of use; second, it aims at identifying conditions that are necessary but not necessarily sufficient for evaluation use; third, it combines mechanisms of evaluation use, context conditions, and actor perceptions. The study reported here examines the use of 11 program and project evaluations by the Swiss Agency for Development and Cooperation (SDC). The article makes use of qualitative comparative analysis (QCA), a method that is well suited to the study of context-bound necessity. It is concluded that the analysis of conditions that are necessary to trigger mechanisms of evaluation use in certain contexts is challenging, but promising to face the complexity of the phenomenon.

Institutional design and utilisation of evaluation results in Uganda’s public universities: Empirical findings from Kyambogo University

“…An attempt to trace its roots as a profession brings about the 'chicken-egg dilemma' because there is hardly any science of what comes first (Basheka & Byamugisha 2015). Ledermann (2012) equally reminds his audience that the questions of whether or not evaluations are used are as old as the evaluation business itself, and this serves to confirm that it is indeed an old practice but a new discipline as more scholarship on the same is relatively new. Patton (2002) suggests that the history of evaluations and their utilisation were holy processes in the management cycle of organisations, arguing that from the bible, the first evaluation of the 10 days dietary project that got Daniel, Hannaniah, Mishael and Azariah fed on vegetables and water healthier and better fed saw its findings put to practical use (Dan 1:1-20).…”

“…The extent to which these evaluations are utilised has been associated with the design of the institutions for which and in which evaluations are carried out. On this subject, Balthasar (2006, 2007in Ledermann 2012, 2009), has systematically presented the effect of the institutional design on the utilisation of evaluations. In addition, for the case of Kyambogo University (KYU), notwithstanding the numerous evaluations that have been carried out, available evidence indicates that the utilisation level for the evaluation results is still weak.…”

“…This low utilisation level has been blamed on the institutional design. Therefore, through this study, the researchers intended to build on the work of Balthasar (2006, 2007in Ledermann 2012, 2009) to examine the effect of the institution's procedural rules, processes and capacities on the utilisation of evaluations at KYU.…”

La investigación evaluativa en el siglo XXI: un instrumento para el desarrollo educativo y social cada vez más relevante

“…Para fortalecer una adecuada utilización de las evaluaciones, es importante analizar bien los contextos de decisión de los usuarios potenciales y centrar el estudio en aspectos relevantes para dicha tarea (Ledermann, 2012;Leviton, 2003;Neuman et al, 2013). El desarrollo de la cultura evaluativa en las organizaciones es otro elemento facilitador de la utilización adecuada de los procesos evaluativos (Cousins et al 2014).…”

Can programme theory be used as a 'translational tool’ to optimise health service delivery in a national early years’ initiative in Scotland: a case study

“…In light of the myriad of factors known to impact on evaluation use, including stakeholders’ receptiveness to findings and perceived need for action [17,18], political contexts [3,17], perceived quality of evaluations [17,18], methods used to communicate findings [17], and timeliness of findings [17], this 'translational gap’ is perhaps unsurprising.…”