Recent interest has been noted in the evaluation community in expanding the focus from program implementation and outcomes to program design and planning. One important step for moving in this direction is to examine existing evaluation models and to assess their relative strengths and weaknesses for planning purposes. This article presents a comparative case study of applying logic models and the action model/change model schema for planning the Learning Community Program in Taiwan. Lessons learned from these applications indicate that logic models are relatively easy to learn and effective for identifying major program components and indicators, but not sufficient for articulating the theoretical significance of the program. On the other hand, the action model/change model schema requires more time to learn and practise, but it has relative advantages for providing theoretical insights into contextual factors and causal mechanisms of the program, unlike logic models. This comparison can serve as a guide for evaluation practitioners when selecting evaluation tools to apply in planning and/or evaluating their programs. Résumé : De plus en plus les évaluateurs semblent orienter leur intérêt vers la conception et la planification de programme plutôt que sur l'implantation et les effets. Une étape importante de ce virage est d' examiner les modèles d' évaluation existants et d' évaluer leurs points forts et leurs points faibles pour l' exercice de planification. Cet article présente une étude de cas comparative de l'utilisation des modèles logiques et des modèle d' action/de changement aux fins de la planification d'un programme d' apprentissage communautaire à Taiwan. Nos résultats indiquent que les modèles© 2018 Canadian Journal of Program Evaluation / La Revue canadienne d'évaluation de programme 33.1 (Spring / printemps), 49-68 figure 1. Action Model/Change Model Schema Using Logic Models and the Action Model/Change Model Schema 57 CJPE 33.1, 49-68 variety of items. This is especially true for the activity component. It seems to me anything we do can be put in this component. In this case, the activity component could become a garbage can. I think the schema overcomes this problem by differentiating components between the action model and change model.

Introduction:Physicians assume leadership roles in their health care organizations and practices often without support or training. The Physicians Leadership Academy provides integrated leadership seminars, mindfulness training, and executive coaching in a 10-month curriculum to physicians across area organizations.Methods:Program evaluators responded to stakeholders' need for continuous program improvement by developing a continuous feedback loop evaluation design incorporating a program monitoring system and a theory-driven program evaluation. Given the size of the 2019 to 20 cohort (n = 19), a one-group pretest/posttest design was used to assess the mechanisms of the program (mindfulness and wellbeing) along with knowledge development, emotional intelligence, and personal and professional growth. The assessments used a combination of published and administrator-developed assessments to address the unique aspects of the program. Doing such ensured continuous improvement and sustainability for the program.Results:The cohort of physicians demonstrated significant engagement and learning across the curriculum, improved mindfulness, and improved capacity of the providers to affect their health care system and communities.Conclusions:The utility of the program was demonstrated through quantitative and qualitative analyses. Implications of the methodology for future evaluations of program developments are discussed.

This study was a formative evaluation of a retention program for nursing students from disadvantaged backgrounds underrepresented in the nursing workforce. The program was challenging, with multilevel interventions, some implemented sequential and some concurrently. The program team was interested to identify problematic components and address them quickly, as part of implementing a rapid cycle improvement process to ensure program success. Evaluators proposed a novel three-wave formative evaluation to meet this need. Results indicated high satisfaction with the program as well as several unintended consequences related to clinical rotations and interaction with faculty and nonparticipant students. The contribution of this study is demonstrating a practical application of an innovative evaluation approach and tool, to assist program planning and to inform remedial actions in a timely manner. Lessons learned from this study contribute to explaining the dynamic nature of unintended effects of program components in the real world and their assessment.

No abstract

No abstract

No abstract

Intervention programs are open systems that must withstand the disturbances of the environments in which they are implemented if they are to thrive. Stakeholders can benefit from proactive insights related to potential program malfunction or failure. Evaluators have not developed approaches to serve this preventive function. This study proposes the program stress test as a holistic, preventive approach to program evaluation, aiming to identify vulnerable areas at risk of failure, and to suggest preventive strategies. The stress test can be used in the planning stage of a program (called ex-ante stress test), or during the implementation stage (called ex-post stress test). This study illustrates an ex-post test. The authors introduce its concept, methodology, research steps, and application to a student retention program, including findings of structural and functional issues, as well as solutions proposed by participants. The authors call the evaluators’ attention to the need to expand the traditional scope of evaluation, to include future-focused assessments.

No abstract