Program evaluation models and related theories: AMEE Guide No. 67

Abstract

This Guide reviews theories of science that have influenced the development of common educational evaluation models. Educators can be more confident when choosing an appropriate evaluation model if they first consider the model's theoretical basis against their program's complexity and their own evaluation needs. Reductionism, system theory, and (most recently) complexity theory have inspired the development of models commonly applied in evaluation studies today. This Guide describes experimental and quasi-experimental models, Kirkpatrick's four-level model, the Logic Model, and the CIPP (Context/Input/ Process/Product) model in the context of the theories that influenced their development and that limit or support their ability to do what educators need. The goal of this Guide is for educators to become more competent and confident in being able to design educational program evaluations that support intentional program improvement while adequately documenting or describing the changes and outcomes-intended and unintended-associated with their programs.

What is the impact of multi-professional emergency obstetric and neonatal care training?

“…Kirkpatrick's model of programme evaluation [73] is one of the most popular systems for evaluating training programmes [74,75]. A number of studies in the literature reviewed indicated how their authors had evaluated their own programmes according to the four levels of training achievements described by Kirkpatrick [11,14,19,50,63,64,68,76].…”

Role-Play Preceded by Fieldwork in the Teaching of Pharmacology: from “Raw Sap” to “Elaborated Sap”

“…The absence of a control group and the fact that the role-play is considered for evaluation in the discipline may represent a selection bias. Moreover, only the students' satisfaction or reaction and some cognitive or behavioural skills were measured, which places the program evaluation on Kirkpatrick's second-level 10 .After the role-play presentation, the acquired pharmacotherapeutic skills were not assessed, for example, in relation to standardized patients (corresponding to Kirkpatrick's third-level) or in different circumstances in the later stages of the course (corresponding to Kirkpatrick's fourth-level). The impact of the intervention on professional practice could not be evaluated, thus further studies are needed to verify the impact of a long-term intervention.…”

La evaluación como instrumento de formación para el aprendizaje a través de los laboratorios remotos

“…Entre otros se indican modelos de evaluación convencionales como el de Kirkpatrick (Boud, et al, 2014;Frye y Hemmer, 2012), el modelo de evaluación de los materiales educativos adaptados a las tecnologías como el de Galvis o Cataldi (Cova, Arrieta y Aular de Duran, 2008;Cataldi, Dominighini, Chiarenza y Lage, 2012), o Bostock que propone una Lista de Control para la Evaluación de Software Educativo (Cova, et al, 2008). Y modelos o investigaciones realizadas donde se pone de manifiesto que la evaluación es una herramienta de formación, como las realizadas por Pineda et al (2014aPineda et al ( , 2014b.…”