Programmatic assessment of competency-based workplace learning: when theory meets practice

Abstract

BackgroundIn competency-based medical education emphasis has shifted towards outcomes, capabilities, and learner-centeredness. Together with a focus on sustained evidence of professional competence this calls for new methods of teaching and assessment. Recently, medical educators advocated the use of a holistic, programmatic approach towards assessment. Besides maximum facilitation of learning it should improve the validity and reliability of measurements and documentation of competence development. We explored how, in a competency-based curriculum, current theories on programmatic assessment interacted with educational practice.MethodsIn a development study including evaluation, we investigated the implementation of a theory-based programme of assessment. Between April 2011 and May 2012 quantitative evaluation data were collected and used to guide group interviews that explored the experiences of students and clinical supervisors with the assessment programme. We coded the transcripts and emerging topics were organised into a list of lessons learned.ResultsThe programme mainly focuses on the integration of learning and assessment by motivating and supporting students to seek and accumulate feedback. The assessment instruments were aligned to cover predefined competencies to enable aggregation of information in a structured and meaningful way. Assessments that were designed as formative learning experiences were increasingly perceived as summative by students. Peer feedback was experienced as a valuable method for formative feedback. Social interaction and external guidance seemed to be of crucial importance to scaffold self-directed learning. Aggregating data from individual assessments into a holistic portfolio judgement required expertise and extensive training and supervision of judges.ConclusionsA programme of assessment with low-stakes assessments providing simultaneously formative feedback and input for summative decisions proved not easy to implement. Careful preparation and guidance of the implementation process was crucial. Assessment for learning requires meaningful feedback with each assessment. Special attention should be paid to the quality of feedback at individual assessment moments. Comprehensive attention for faculty development and training for students is essential for the successful implementation of an assessment programme.

Students’ motivation toward feedback-seeking in the clinical workplace

“…In the context of the applied theoretical model of longitudinal programmatic assessment at the Faculty of Veterinary Medicine, Utrecht University, every individual assessment is low stakes/formative. However, earlier research showed that in the practical implementation of this theoretical framework, students do not seem to perceive individual assessments as formative, but rather as summative (Bok et al 2013b). The findings in this study might suggest that some students -the high performing ones -do see the beneficial effects of feedback in developing themselves and are more likely to use the feedback in a rather formative way.…”

Estruturação de Sistemas para Avaliação Programática do Estudante de Medicina

“…A educação e o treinamento do estudante de Medicina compõem um programa formativo, sendo assim também pertinente conceber a avaliação do estudante como tendo uma dimensão programática. Deste modo, pode-se conceituar a avaliação programática como um conjunto de ações diversificadas, mas coordenadas, que se desenvolvem em diferentes momentos, compondo um verdadeiro programa, visando ao cumprimento, da melhor forma possível, das várias funções da avaliação do estudante [5][6][7][8][9][10][11] . Em outras palavras, pode-se entender a avaliação programática do estudante como uma forma inovadora de aplicar conceitos tradicionais, para que as funções da avaliação sejam cumpridas com eficácia (fitness-for-purpose).…”

Evaluación de competencias prácticas clínicas por el alumnado y el profesorado a través del portafolios

“…Uno de los instrumentos de evaluación más utilizado en la construcción del espacio común de educación superior en todas las ramas de conocimiento, y de forma más concreta en Ciencias de la Salud, es el portafolios (Saltman Tavabie y Kidd, 2012;Vaughan et al, 2012;Forte, de Souza, da Silva, do Prado y Rodrigues, 2013;Vance et al, 2013;Chiu Lee y Ho, 2014;Haldane, 2014), especialmente por su capacidad para evaluar las competencias desarrolladas por el estudiante (Suraseranivongse et al, 2011;Cheung, 2011;Atahawi et al, 2012;Webb et al, 2014) y por el protagonismo que otorga al alumnado en su proceso de aprendizaje y evaluación (Finlay, Probyn y Ho, 2012;Bok et al, 2013). No obstante, a pesar de las ventajas que conlleva su utilización, no está exento de dificultades (Driessen Van Tartwijk, Van Der Vleuten y Wass, 2007).…”

“…La reforma universitaria europea defiende un modelo educativo "competencial" en el que las decisiones sobre el currículo (especialmente las relativas a los contenidos, la metodología y evaluación) se realizan en función de las competencias que los estudiantes deben demostrar al finalizar la titulación cursada (Biggs, 2004;Colás, Jiménez y Villaciervos, 2005;Martínez y Echeberría, 2009; Atahawi, Sisk, Poloskey, Hicks y Dannefer, 2012;Bok et al, 2013).…”