A randomised controlled pilot trial of the influence of non-native English accents on examiners’ scores in OSCEs

Abstract

Background Objective structured clinical examinations (OSCEs) are important aspects of assessment in medical education. There is anecdotal evidence suggesting that students with non-native English accents (NNEA) may be subjected to unconscious bias. It is imperative to minimise the examiners’ bias so that the difference in the scores reflects students’ clinical competence. Research shows NNEAs can cause stereotyping, often leading to the speaker being negatively judged. However, no medical education study has looked at the influence of NNEAs in assessment. Methods This is a randomized, single-blinded controlled trial. Four videos of one mock OSCE station were produced. A professional actor played a medical student. Two near identical scripts were prepared. Two videos showed the actor speaking with an Indian accent and two videos showed the actor speaking without the accent in either script. Forty-two OSCE examiners in the United Kingdom (UK) were recruited and randomly assigned to two groups. They watched two videos online, each with either script, each with a different script. One video with a NNEA and one video was without. Checklist item scores were analysed with descriptive statistics and non-parametric independent samples median test. Global scores were analysed with descriptive statistics and Mann-Whitney test. Results Thirty-two examiners completed the study. The average scores for the checklist items (41.6 points) did not change when the accent variable was changed. Independent samples median test showed no statistically significant relationship between the accent and the scores ( p = 0.787). For the global scores received by the videos with the NNEA, there were one less ‘Good’ grade and one more ‘Fail’ grade compared to those without the NNEA. Mann-Whitney test on global score showed lower scores for videos with NNEA ( p = 0.661). Conclusions Examiners were not biased either positively or negatively towards NNEAs when providing checklist or global scores. Further study is required to validate the findings of this study. More discussion is warranted to consider how the accent should be considered in current medical education assessment. Registration Trial registration completed trial, ID: ISRCTN17360102, Retrospectively registered on 15/04/2020.

Formando médicos para el siglo XXI: El rol crucial de la educación basada en competencias

“…Tradicionalmente, los exámenes médicos se han centrado en evaluar el rendimiento de los estudiantes con respecto a unos estándares predeterminados en un momento determinado como evaluación sumativa (46) . Sin embargo, en los planes de estudios basados en las competencias, la evaluación formativa se considera más adecuada, ya que orienta el aprendizaje posterior (47)(48) .…”

How do field epidemiologists learn? A protocol for a qualitative inquiry into learning in field epidemiology training programmes

“…In contrast, articles describing learning processes in medicine and nursing abound 19–24. To take one example, Ohta et al used grounded theory to reveal learning processes in Japanese medical students’ transitions to rural community hospitals 25.…”

How Do Field Epidemiologists Learn? A Protocol for a Qualitative Inquiry Into Learning in Field Epidemiology Training Programs

“…(18) In contrast, articles describing learning processes in medicine and nursing abound. (19)(20)(21)(22)(23)(24) To take one example, Ohta and colleagues utilized grounded theory to reveal learning processes in Japanese medical students' transitions to rural community hospitals. (25) They found that integration of cognitive apprenticeship (26) and legitimate peripheral participation in communities of practice (27) with learners' regular reflection on performance facilitated their learning family medicine.…”