Student assessment by objective structured examination in a neurology clerkship

Abstract

Objectives:We evaluated the reliability and predictive ability of an objective structured clinical examination (OSCE) in the assessment of medical students at the completion of a neurology clerkship. Methods:We analyzed data from 195 third-year medical students who took the OSCE. For each student, the OSCE consisted of 2 standardized patient encounters. The scores obtained from each encounter were compared. Faculty clinical evaluations of each student for 2 clinical inpatient rotations were also compared. Hierarchical regression analysis was applied to test the ability of the averaged OSCE scores to predict standardized written examination scores and composite clinical scores.Results: Students' OSCE scores from the 2 standardized patient encounters were significantly correlated with each other (r ϭ 0.347, p Ͻ 0.001), and the scores for all students were normally distributed. In contrast, students' faculty clinical evaluation scores from 2 different clinical inpatient rotations were uncorrelated, and scores were skewed toward the highest ratings. After accounting for clerkship order, better OSCE scores were predictive of better National Board of Medical Examiners standardized examination scores (R 2 ⌬ ϭ 0.131, p Ͻ 0.001) and of better faculty clinical scores (R 2 ⌬ ϭ 0.078, p Ͻ 0.001). There is a developing body of research on clinical neurology education. 1 One goal in neurology education is improved student assessment using valid objective methods demonstrating consistency and also providing well-distributed scores, avoiding the grade inflation prevalent with faculty evaluations. Semiobjective systems using nonstandardized patients to evaluate performance on neurology clerkships have been reported. These have shown, however, limited ability to predict subjective clinical scores in neurology. Conclusions:2 An alternative approach is use of an objective structured clinical examination (OSCE) in neurology.3-6 The OSCE involves a series of encounters objectively testing clinical competencies including history taking, physical examination, communication, and data interpretation. It can be both a formative and summative evaluation.7 Feasibility of use of an OSCE in the examination of medical students at the completion of a neurology clerkship has been reported, 3,4,6,8 -10 but assessment of its reliability as an evaluation tool in a neurology clerkship has not been described.We recently implemented an OSCE as a supplementary tool for the objective assessment of clinical skills of medical students completing the neurology clerkship. We assessed the parallel forms reliability of this OSCE by comparing student performance across 2 separate standardized patient (SP) experiences. We also tested the OSCE's ability to predict outcomes on other

Neurology objective structured clinical examination reliability using generalizability theory

“…Students are randomly assigned to 2 of 3 cases, and thus calculations were made for each possible combination of cases (cases 1 and 2, cases 2 and 3, cases 1 and 3), as well as a "combined" calculation that represents an aggregate of estimates across all case combinations. 8 These results are helpful in arguing to include an OSCE as a component of students' assessment portfolios within a neurology clerkship. Educators would need to double the number of cases per student (3)(4)(5)(6) in order to increase the reliability by a tenth of a point, but only one additional case is needed to reach the desired threshold.…”

Exame Clínico Estruturado Como Ferramenta Educacional Na Área De Saúde: Revisão Integrativa

“…A confiabilidade geral do ECOE foi satisfatória quando confrontada com estudos similares (26) . (19) . De modo a reforçar esse achado, ressalta-se o estudo desenvolvido nos Estados Unidos com alunos de graduação, que utilizou o ECOE para desenvolver o raciocínio clínico sobre o sistema musculoesquelético com base nos princípios do exame físico orientando a hipótese de diagnóstico.…”

Formal faculty observation and assessment of bedside skills for 3rd-year neurology clerks

“…5 In neurology, there has been an increase in objective structured clinical examinations (OSCEs) in medical students' education, but feedback is primarily provided by standardized patients rather than faculty. 6,7 Few studies have examined the feasibility and utility of medical student bedside clinical evaluations during the neurology clerkship. Schmahmann et al 8 reported on an experience with a beside examination exercise and Lukas et al 6 described utilizing an OSCE, but neither approach included direct faculty observation of students conducting a full history and From the University of Rochester, Rochester, NY.…”

“…6,7 Few studies have examined the feasibility and utility of medical student bedside clinical evaluations during the neurology clerkship. Schmahmann et al 8 reported on an experience with a beside examination exercise and Lukas et al 6 described utilizing an OSCE, but neither approach included direct faculty observation of students conducting a full history and From the University of Rochester, Rochester, NY.Go to Neurology.org for full disclosures. …”