Background and history of the Joint Committee's Program Evaluation Standards

BackgroundQuality circles (QCs) are commonly used in primary health care in Europe to consider and improve standard practice over time. They represent a complex social intervention that occurs within the fast-changing system of primary health care. Numerous controlled trials, reviews, and studies have shown small but unpredictable positive effect sizes on behavior change. Although QCs seem to be effective, stakeholders have difficulty understanding how the results are achieved and in generalizing the results with confidence. They also lack understanding of the active components of QCs which result in changes in the behavior of health care professionals. This protocol for a realist synthesis will examine how configurations of components and the contextual features of QCs influence their performance.Methods/DesignStakeholder interviews and a scoping search revealed the processes of QCs and helped to describe their core components and underlying theories. After clarifying their historical and geographical distribution, a purposive and systematic search was developed to identify relevant papers to answer the research questions, which are: understanding why, how, and when QCs work, over what time frame, and in what circumstances. After selecting and abstracting appropriate data, configurations of contexts and mechanisms which influence the outcome of QCs within each study will be identified. Studies will be grouped by similar propositional statements in order to identify patterns and validation from stakeholders sought. Finally, theories will be explored in order to explain these patterns and to help stakeholders maintain and improve QC performance.DiscussionAnalyzing context-mechanism-outcome (CMO) patterns will reveal how QCs work and how contextual factors interact to influence their outcome. The aim is to investigate unique configurations that enable them to improve the performance of health care professionals. Using a standardized reporting system, this realist review will allow the research questions to be answered to the satisfaction of key stakeholders and enable on-going critical examination and dissemination of the findings.Study registrationPROSPERO registration number: CRD42013004826.

show abstract

“…However, it will follow the relevant standards of utility, usefulness, feasibility, propriety, accuracy, and accountability [70,71]. …”

Section: Discussionmentioning

confidence: 99%

Exploring why quality circles work in primary health care: a realist review protocol

2013

View full text Add to dashboard Cite

show abstract

“…To facilitate this purpose, the procedures of the American National Standards Institute (ANSI) and Joint Committee on Standards for Educational Evaluation (JCSEE) required that all approved standards undergo a major revision process that included current stakeholders at least every 10 years. Because the second edition was completed in 1994, the standards were out of date without revision by ( JCSEE, 1981, 1988, 1994Yarbrough, Shulha, & Caruthers, 2004 ;cf. Yarbrough et al, 2011 ).…”

mentioning

confidence: 99%

“…When aligned with scholarship, the standards can then serve as effi cient scaff olding for knowledge and values related to evaluation and help facilitate better evaluation practice (cf. Yarbrough et al, 2004 ). Much of this article is directed to documenting how the scholarship on evaluation use, utility, and collaboration provided a foundation for the utility standards development processes and the products that resulted (cf.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Developing the Program Evaluation Utility Standards: Scholarly Foundations and Collaborative Processes

Yarbrough

2017

Canadian Journal of Program Evaluation

Self Cite

View full text Add to dashboard Cite

Developing the third edition of the program evaluation utility standards required multilevel collaborations among task force members, members of the Joint Committee on Standards for Educational Evaluation, sponsoring organizations, and hundreds of involved stakeholders. The scholarship on evaluation use, influence, and collaboration was foundational for the utility standards and materials accompanying them and equally important for informing the processes guiding utility standards development. This article emphasizes the foundational role of this recent scholarship and the roles played by all who collaborated in planning and implementing the utilitystandards development processes.Il a fallu une collaboration à toutes sortes de niveaux entre les membres du groupe de travail, les membres du Comité mixte sur les normes d'évaluation en éducation, les organisations commanditaires et des centaines d’intervenants pour arriver à la troisième édition des normes d’utilité en évaluation de programme. Les normes d’utilité et les documents connexes ont leur fondement dans les recherches sur l’utilisation, l’influence de l’évaluation, et la collaboration, recherches qui ont eu un impact important sur les processus qui ont guidé l’établissement des normes d’utilité. Le présent article met l’accent sur le rôle fondamental de ces recherches récentes et sur les rôles joués par toutes les personnes qui ont contribué à la planifi cation et à la mise en oeuvre des processus d’élaboration des normes d’utilité.

show abstract

“…It is unlikely, unless of course we have written popular books or advertise ourselves as practitioners with a specialization in a particular approach, that we will be known in the program community by any of the CPE labels. It is in establishing credibility, attending to stakeholders, and negotiating evaluation purposes-that is, the first three of the revised ''Program Evaluation Standards'' (Yarbrough, Shulha, Hopson, & Caruthers, 2011)-that most evaluators, with their clients, will determine the best approach for the evaluation challenge at hand.…”

mentioning

confidence: 99%

Let There Be Light

Cousins

Whitmore

Shulha

2013

American Journal of Evaluation

View full text Add to dashboard Cite

make a plea for ''more light than heat.'' On that front, we are completely in accord. Indeed, in many respects, we find ourselves on the same side of the coin when it comes to collaborative inquiry in evaluation. Yet, as we lay out in this response to their letter, there are some points of departure that we wish to highlight because we do have differences and we believe it is important to be clear about them. Let us elaborate.On the other side of the aforementioned coin are those in our field, who argue for the supremacy of traditional objectivist forms of evaluation that view stakeholder participation in the production of evaluation knowledge as a form of heresy. Stufflebeam's (1994) polemic ''Where the future of evaluation should not go and where it needs to go'' is a shining example. That critique was written in response to Fetterman's 1993 American Evaluation Association (AEA) presidential address that extolled the virtues of a new approach in our field, empowerment evaluation. Fetterman's term as president was followed by the establishment of collaborative, participatory, and empowerment (CPE) evaluation that has grown to become AEA's sixth most subscribed topical interest groups (TIGs). 1 But collaborative inquiry in evaluation, and more generally in the social sciences, did not start with empowerment evaluation or with the establishment of the CPE-TIG. It started a good number of years earlier in international development toward the late 1970s (Brisolara, 1998), and it had a particularly democratizing and transformative flavor to it. Over time many and varied species emerged in both developing and western contexts including, but by no means limited to, participatory action research (PAR), democratic evaluation, educative research, participatory rural appraisal (PRA), illuminatory evaluation, emancipatory and critical action research, collaborative action research, teacher research, deliberative democratic evaluation, and developmental evaluation (see Cousins & Chouinard, 2012, pp. 19-21 for elaboration). Of course, we need to add to the list CPE approaches, but what sets this group apart, at least in the words of Fetterman and associates, is the propensity for self-identification with the respective approach. ''Collaborative evaluators are . . . .

show abstract

Background and history of the Joint Committee's Program Evaluation Standards

Abstract: To ground the discussion of standards in an international context, this chapter traces the continuing history of the U.S.‐Canadian Joint Committee Standards.

Cited by 108 publications

References 26 publications

Exploring why quality circles work in primary health care: a realist review protocol

Exploring why quality circles work in primary health care: a realist review protocol

Developing the Program Evaluation Utility Standards: Scholarly Foundations and Collaborative Processes

Let There Be Light

Contact Info

Product

Resources

About