A Research Synthesis of the Evaluation Capacity Building Literature

Abstract

The continuously growing demand for program results has produced an increased need for evaluation capacity building (ECB). The Integrative ECB Model was developed to integrate concepts from existing ECB theory literature and to structure a synthesis of the empirical ECB literature. The study used a broad-based research synthesis method with systematic decision rules and demonstrates the viability of the method for producing a reliable analysis of disparate data from a variety of designs. There was a high degree of consistency in what was reported in the empirical literature and the theoretical literature in terms of strategies and outcomes. Reported outcomes at the individual level included attitudes, knowledge, and behaviors and at the organizational level included practices, leadership, culture, mainstreaming, and resources. Collaborative processes and programmatic outcomes emerged as important issues for ECB models and practice. The consistency between the empirical and the theoretical literature indicates that the field is ready to develop common measures, use stronger designs, and report more systematically. This synthesis provides an overview of existing data and an empirical basis for refining strategies and common measures for enhancing the research and practice of ECB to achieve ECB and programmatic goals and outcomes.

Strategic Evaluation Utilization in the Canadian Federal Government

“…The three dimensions that are of particular conceptual interest for the present study are those related to organizational capacity to use evaluation: 1) Evaluation Literacy refers to the broad knowledge of evaluation across the organization; 2) Organizational Decision-Making focuses on the instrumental and conceptual application of evaluation findings in broader processes, such as program development or budget reallocation exercises; and 3) Learning Benefits focuses on applying knowledge derived from evaluations into everyday activities and organizational learning (including instrumental and process uses). It should be noted that most of the research conducted in the area of evaluation capacity focuses on evaluation's supply side (capacity to do evaluation), and little attention has been paid to its demand side (capacity to use evaluation): for example, models developed by Preskill and Boyle (2008) and Labin, Duffy, Meyers, Wandersman, and Lesesne (2012), as well as others, mainly identify practices and behaviours associated with organizational capacity to do evaluation rather than use (Bourgeois, 2016). This study, therefore, offers a unique perspective focused primarily on organizational capacity to use evaluation, and how this capacity effectively translates into concrete utilization.…”

Construction of Indicators of the Self-evaluation Capacity Building through Focused Group for Technical High Schools in Taiwan

“…It is found in the literature review that there are 5 theories of evaluation capacity building: evaluation capacity building pathway model (ECBP) [7], evaluation capacity building measure model (ECBM) [8], evaluation capacity building of multidisciplinary model (ECBMP) [9], integrate model of evaluation capacity building (IECB) [10], and the expanded IECB model (EIECB) [10].…”

“…But there are some differences in the meaning at the same time, for instance, some scholars emphasize that ECB is a routine in an organization [5], while some believe that under the exploration of the dimension of individual and organization, no matter individual or organization need to conduct capacity building [9]. Labin, Duffy, Meyers, Wandersman and Lesesne suggest that evaluation capacity building (ECB) is a process to enhance personal motive, knowledge, and skills and to strengthen groups or organizations to conduct or use evaluation capability [10]. Figure 3.…”

The Unified Outcomes Project: Evaluation Capacity Building, Communities of Practice, and Evaluation Coaching

“…This article focuses on two specific mechanisms that facilitated these outcomes: communities of practice (CP) and communities of practice with coaching (CPC). Multiple ECB models (Preskill & Boyle, 2008;Labin, et al, 2012) note that a combination of ECB strategies, including coaching and CP, are associated with higher levels of organizational outcomes. In comparison to previous case examples (Arnold, 2006;Stevenson, Florin, Mills, & Andrade, 2002;Taut, 2007;Ensminger, Kallemeyn, Rempert, Wade, & Polanin, 2015), the Unified Outcomes Project focuses on the mechanisms of CP and CPC to highlight a unique approach to ECB that could potentially be used across various foundation contexts.…”

“…Key principles include a continuous back-and-forth between parts and the whole to make meaning, such as experiences of one grantee in the relation to the entire sample; a goal of reaching inner unity in the findings; awareness that the researchers influence the interpretations; and the importance of the interpretations promoting innovation and new directions. During this process, the research team applied ECB frameworks (Preskill & Boyle, 2008;Labin, et al, 2012) and allowed for emergent themes. Frequent meetings were held to gain consensus among the research team, evaluation coach, foundation staff, and selected participants.…”