This article introduces a set of evidence-based principles to guide evaluation practice in contexts where evaluation knowledge is collaboratively produced by evaluators and stakeholders. The data from this study evolved in four phases: two pilot phases exploring the desirability of developing a set of principles; an online questionnaire survey that drew on the expertise of 320 practicing evaluators to identify dimensions, factors or characteristics that enhance or impede success in collaborative approaches in evaluation (CAE); and finally a validation phase involving a subsample of 58 evaluators who participated in the main phase. The principles introduced here stem from the experiences of evaluators who have engaged in CAE in a wide variety of evaluation settings and contexts and the lessons they have learned. They are understood to be interconnected and loosely temporally ordered. We expect the principles to evolve over time, as evaluators learn more about collaborative approaches in context. With this in mind, we pose questions for consideration to stimulate further inquiry.
Evaluation policy has been identified as an important means of shaping and influencing organizational evaluation practice, yet, to date, little empirical research has been conducted to deepen our understanding of this relationship. The purpose of this study was to illuminate evaluation policy’s role in leveraging organizational capacity to do and use evaluation. We interviewed 18 published evaluation scholars and practitioners from North America and Europe about this topic. A thematic analysis of findings underscores the importance of context, policy attributes, enablers, and organizational benefits. Based on the findings, we developed an ecological conceptual framework to guide thinking about the role of evaluation policy in capacity building. We discuss these findings in terms of practical implications of understanding context, redressing the imbalance between learning and accountability purposes of evaluation, and organizational leadership, and we conclude with some implications for research.
How do evaluators using collaborative approaches to evaluation (CAE) define success? This is the core question being asked in a further analysis of data from our previous work ( Cousins, Whitmore, & Shulha, 2013 ; Shulha et al., 2016) that developed a set of evidence-based principles to guide collaborative evaluation practice. Probing data from 320 responses to our (2012) survey, we examined what respondents considered “highly successful” and “less successful than hoped” in their collaborative evaluation projects. The results revealed that evaluation use, relationships, and information needs are key factors. We propose a conceptual framework as an aid to thinking about success in CAE.Comment les évaluateurs utilisant des approches collaboratives à l’évaluation définissent-ils le succès? Voici la question de base posée dans une analyse plus poussée de données tirées de travaux précédents (Cousins, Whitmore et Shulha, 2013; Shulha et al., 2016) qui ont permis d’élaborer un ensemble de principes scientifi quement fondés visant à orienter la pratique de l’évaluation collaborative. En examinant les données de 320 réponses à notre sondage (de 2012), nous nous sommes penchés sur ce que nos répondants ont jugé être des projets d’évaluation collaborative « très réussis » et « moins réussis qu’espéré ». Les résultats ont révélé que l’utilisation de l’évaluation, les relations et les besoins en information sont des facteurs clés. Nous proposons un cadre conceptuel pour penser le succès en matière d’approche collaborative en évaluation.
Through a guided discussion, this article explores a five-year cross-cultural evaluation relationship comprising multiple projects involving an evaluator from Canada and a group of Indian colleagues working on educational reform in India. The initiative was funded through a multilateral consortium of donors and involved Western evaluation specialists working in collaboration with Indian colleagues to (a) develop evaluation capacity within the country and (b) produce evaluative knowledge about education quality initiatives associated with large-scale educational reform. This article is based on a conversation between the principal investigator from Canada and three Indian colleagues who had been involved in all phases of the work. It focuses on their respective perspectives and experiences, including the benefits obtained and the challenges encountered in the process of bridging Western and Indian knowledge systems. The article begins with background about the initiative and continues with a conversation among the participants about their cross-cultural evaluation experience. It concludes with an analysis of the issues that emerged and generation of lessons learned for evaluators interested in cross-cultural evaluation.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.