Comparison of student examiner to faculty examiner scoring and feedback in an OSCE

Abstract

OBJECTIVES To help reduce pressure on faculty staff, medical students have been used as raters in objective structured clinical examinations (OSCEs). There are few studies regarding their ability to complete checklists and global rating scales, and a paucity of data on their ability to provide feedback to junior colleagues. The objectives of this study were: (i) to compare expert faculty examiner (FE) and student-examiner (SE) assessment of students' (candidates') performances on a formative OSCE; (ii) to assess SE feedback provided to candidates, and (iii) to seek opinion regarding acceptability from all participants.METHODS Year 2 medical students (candidates, n = 66) participated in a nine-station formative OSCE. Year 4 students (n = 27) acted as SEs and teaching doctors (n = 27) served as FEs. In each station, SEs and FEs independently scored the candidates using checklists and global rating scales. The SEs provided feedback to candidates after each encounter. The FEs evaluated SEs on the feedback provided using a standardised rating scale (1 = strongly disagree, 5 = strongly agree) for several categories, according to whether the feedback was: balanced; specific; accurate; appropriate; professional, and similar to feedback the FE would have provided. All participants completed questionnaires exploring perceptions and acceptability.RESULTS There was a high correlation on the checklist items between raters on each station, ranging from 0.56 to 0.86. Correlations on the global rating for each station ranged from 0.23 to 0.78. Faculty examiners rated SE feedback highly, with mean scores ranging from 4.02 to 4.44 for all categories. There was a high degree of acceptability on the part of candidates and examiners.CONCLUSIONS Student-examiners appear to be a viable alternative to FEs in a formative OSCE in terms of their ability to both complete checklists and provide feedback.

A student-initiated objective structured clinical examination as a sustainable cost-effective learning experience

“…Cushing and colleagues [15] have also described mixed findings in this regards; in their OSCE, some examinees felt that having their classmates be their peer assessors helped them relax whereas others perceived increased pressure. In a study by Moineau and colleagues [12], the authors administered a post-OSCE questionnaire to second-year examinees that was adapted for use in the present study; they found that most examinees did not perceive tension with their fourth-year student examiners, and furthermore there were no comments from examinees to suggest the contrary. It is possible that larger gaps in training level between examinees and examiners decrease the potential for awkward or tense interactions.…”

“…Close-ended questions consisted of statements with response options arranged in a five-point Likert scale ranging from ‘Strongly Disagree’ (1) to ‘Strongly Agree’ (5). In the examinee questionnaire, close-ended questions were adapted from Moineau and colleagues [12] (see Table 1). For examiner and SP questionnaires, close-ended questions were based on questionnaires designed by Burgess and colleagues [13] (see Tables 2 and 3, respectively).…”

Examen clínico objetivo y estructurado formativo tras inmersión clínica precoz empleando estudiantes de sexto curso como observadores y administradores de retroalimentación

“…Por ello decidimos adelantar la exposición clínica de nuestros estudiantes al tercer curso del grado recurriendo a un curso de transición del periodo básico al clínico y a rotaciones precoces a expensas del tiempo lectivo inicialmente dedicado a las enseñanzas teóricas. En consonancia con el principio de que la evaluación debe dirigir el aprendizaje, diseñamos un ECOE formativo con el doble propósito de evaluar la efectividad de la inmersión clínica precoz y comprobar si estudiantes apropiadamente entrenados son capaces de dar retroalimentación adecuada y usar escalas globales de estimación de la competencia de una manera fiable, replicando así los resultados del estudio de Moineau et al [8], quienes usaron por primera vez estudiantes senior para dar feedback bajo la supervisión de un profesor.…”

“…En el intervalo entre ambas partes de la estación, el paciente y el estudiante cumplimentaron los listados de ítems (anamnesis-exploración-comunicación; puntuación máxima de 100) y la escala global. Al finalizar el ECOE, observadores y candidatos cumplimentaron cuestionarios adaptados de Moineau et al [8] para recoger sus juicios, percepciones y grado de satisfacción con las tareas asumidas.…”

“…La presión creciente sobre el profesorado, por un lado, y la evidencia proporcionada por algunos estudios de que estudiantes senior convenientemente entrenados pueden asumir estas tareas [8], nos llevó a emplear estudiantes de sexto curso como observadores y proveedores de retroalimentación en un ECOE de propósito principalmente formativo, administrado a estudiantes de tercer curso expuestos a las intervenciones docentes antes mencionadas. El ECOE persiguió como objetivos adicionales estimar el nivel de competencia clínica de los candidatos y familiarizarlos con un tipo de prueba que han de pasar con carácter sumativo en cursos posteriores.…”