Developing evaluation standards and assessing evaluation quality

Leslie, Mike; Moodley, Nishendra; Goldman, Ian; Jacob, Christel; Podems, Donna; Everett, Mark E.; Beney, Terence

doi:10.4102/aej.v3i1.112

Cited by 5 publications

(6 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It has more than 2 million members and has been recognized by 12-15 professional organizations [18]. Recently, the JCSEE program evaluation standards have also been recognized by the authorities, such as the American Evaluation Association, and have been recommended as the guideline for designing other evaluation standards [3,24].…”

Section: Background Of Meta-evaluationmentioning

confidence: 99%

Meta-Evaluation for the Evaluation of Environmental Management: Standards and Practices

Yin

Guo

2021

Sustainability

View full text Add to dashboard Cite

Environmental management plays a key role in the sustainable development of cities. The effectiveness of environmental management is commonly examined through some evaluation schemes, but the effectiveness of such evaluation schemes also needs to be verified. As such, in this study, meta-evaluation was creatively introduced into environmental management to improve the evaluation of environmental management (EEM). Meta-evaluation is the evaluation of an evaluation scheme, and can verify and enhance the evaluation quality. First, a set of new meta-evaluation standards and criteria was proposed based on the unique characteristics of environmental management, which made meta-evaluation standards more adaptable and effective. After that, the efficacy of the proposed meta-evaluation standards was verified through their application to two evaluation schemes used in different fields of EEM. Based on meta-evaluation, suggestions for improving these two EEM schemes were also provided. The major contributions of this study are to introduce meta-evaluation into environmental management, establish new evaluation standards, and examine the efficacy of EEM. The research showed that it is critical to carry out meta-evaluation before and/or after the implementation of EEM.

show abstract

Section: Background Of Meta-evaluationmentioning

confidence: 99%

Meta-Evaluation for the Evaluation of Environmental Management: Standards and Practices

Yin

Guo

2021

Sustainability

View full text Add to dashboard Cite

show abstract

“…The JCSEE Program Evaluation Standards are utility, feasibility, propriety, accuracy, and evaluation accountability. They are supported by the American Evaluation Association (AEA) and have been the basis for the development of other standards (Leslie et al, 2015). Stufflebeam and Coryn (2014) recommend using these standards, along with the Guiding Principles for Evaluators (AEA, 2018), to evaluate an evaluation.…”

Section: Quality Standardsmentioning

confidence: 99%

Internal Formative Meta-Evaluation: Assuring Quality in Evaluation Practice

Harnar

Hillman

Endres

et al. 2020

American Journal of Evaluation

View full text Add to dashboard Cite

The term meta-evaluation—referring to the “evaluation of evaluations”—has been in the evaluation lexicon for a half-century. Despite this longevity, research on meta-evaluation is sparse and even more so for internal formative types of meta-evaluation. This exploratory study builds on our understanding of meta-evaluative methods by exploring evaluators’ approaches to ensuring quality practice. A sample of practitioners was drawn from the American Evaluation Association membership and invited to share their quality assurance practices through an online survey. Respondents reported using a variety of tools to ensure quality in their practice, including published and unpublished standards, principles and guidelines, and processes involving stakeholder engagement at various stages of evaluation. A distinction was identified between an intrinsic, merit-focused perspective on quality that is more or less controlled by the evaluator and an extrinsic, worth-focused perspective on quality primarily informed by key stakeholders of the evaluation.

show abstract

“…A number of other noteworthy developments unfolded in 2012, including (a) the publication of the draft of the South African Standards for Evaluation in Government, (b) the formalization of the DPME–SAMEA partnership (the South African Monitoring and Evaluation Association is a voluntary organization for professional evaluators established in 2005), and (c) a first systematic attempt to capture how M&E systems are faring in 96 national and provincial departments by the DPME in collaboration with the Deutsche Gesellschaft für Internationale Zusammenarbeit (Beney et al, 2015; Leslie et al, 2015; Umlaw and Chitepo, 2015). The findings of the 2012 situational analysis speak to a number of deficiencies in the institutional environment that compromised the implementation of M&E functions.…”

Section: South Africamentioning

confidence: 99%

Evaluation culture and good governance: Is there a link?

Dahler‐Larsen

Boodhoo

2019

Evaluation

View full text Add to dashboard Cite

An important motivation for the institutionalization of evaluation culture in countries around the world is the belief that accountability and transparency will thereby be enhanced. We subject this narrative about evaluation’s contribution to good governance to empirical analysis. We also argue that the meaning and relevance of this general narrative differs across national contexts. We build on data from a systematic assessment of evaluation culture in 19 countries (Jacob et al., 2015), add one country using the same systematic approach, and combine these findings with an indicator of transparency in government provided by Transparency International. We find a positive correlation between evaluation culture and transparency, and discuss threats to a causal interpretation hereof. We go into depth with two particular countries at either end of the transparency scale. We argue that the meaning of the link between evaluation culture and transparency differs whether the chosen perspective is generally comparative or situated in particular national, historical, and political contexts. In countries where transparency is high already, there might be diminishing marginal returns on evaluation, at least regarding its contribution to accountability and transparency.

show abstract

Developing evaluation standards and assessing evaluation quality

Cited by 5 publications

References 1 publication

Meta-Evaluation for the Evaluation of Environmental Management: Standards and Practices

Meta-Evaluation for the Evaluation of Environmental Management: Standards and Practices

Internal Formative Meta-Evaluation: Assuring Quality in Evaluation Practice

Evaluation culture and good governance: Is there a link?

Contact Info

Product

Resources

About