“Shadow” OSCE examiner. A cross-sectional study comparing the “shadow” examiner with the original OSCE examiner format

Abstract

OBJECTIVES:Feedback is a powerful learning tool, but a lack of appropriate feedback is a very common complaint from learners to teachers. To improve opportunities for feedback on objective structured clinical examinations (OSCEs), a modified examiner role, termed the “shadow” examiner, was tested. This study aims to present and analyze comparisons between the “shadow” examiner and the original OSCE examiner format.METHODS:In 2011, experiments were carried out with modifications to the examiner’s role to define the “shadow” examiner format. From February 2012 to May 2014, research was conducted with 415 6th-year medical students. Of these students, 316 were randomly assigned to assessments by both “shadow” and “fixed” examiners. Pearson correlation analysis with linear regression, Student’s t-tests and Bland-Altman plots were the statistical methods used to compare the assessment modes. To strengthen the analysis, checklist items were classified by domain.RESULTS:High correlations between the “shadow” and “fixed” examiners’ global scores were observed. The results of the analysis of specific domains demonstrated higher correlations for cognitive scores and lower correlations for affective scores. No statistically significant differences between the mean examiner global scores were found. The Bland-Altman analysis showed that the “shadow” examiners’ affective scores were significantly higher than those of the “fixed” examiners, but the magnitude of this difference was small.CONCLUSION:The modified examiner role did not lead to any important bias in the students’ scores compared with the original OSCE examiner format. This new strategy may provide important insights for formative assessments of clinical performance.