Can standardized patients replace physicians as OSCE examiners?

Abstract

BackgroundTo reduce inter-rater variability in evaluations and the demand on physician time, standardized patients (SP) are being used as examiners in OSCEs. There is concern that SP have insufficient training to provide valid evaluation of student competence and/or provide feedback on clinical skills. It is also unknown if SP ratings predict student competence in other areas. The objectives of this study were: to examine student attitudes towards SP examiners; to compare SP and physician evaluations of competence; and to compare predictive validity of these scores, using performance on the multiple choice questions examination (MCQE) as the outcome variable.MethodsThis was a cross-sectional study of third-year medical students undergoing an OSCE during the Internal Medicine clerkship rotation. Fifty-two students rotated through 8 stations (6 physician, 2 SP examiners). Statistical tests used were Pearson's correlation coefficient, two-sample t-test, effect size calculation, and multiple linear regression.ResultsMost students reported that SP stations were less stressful, that SP were as good as physicians in giving feedback, and that SP were sufficiently trained to judge clinical skills. SP scored students higher than physicians (mean 90.4% +/- 8.9 vs. 82.2% +/- 3.7, d = 1.5, p < 0.001) and there was a weak correlation between the SP and physician scores (coefficient 0.4, p = 0.003). Physician scores were predictive of summative MCQE scores (regression coefficient = 0.88 [0.15, 1.61], P = 0.019) but there was no relationship between SP scores and summative MCQE scores (regression coefficient = -0.23, P = 0.133).ConclusionThese results suggest that SP examiners are acceptable to medical students, SP rate students higher than physicians and, unlike physician scores, SP scores are not related to other measures of competence.

Validation and Evaluation of an OSCE in Undergraduate Doctor of Pharmacy Program

“…22,23 Although a larger number of stations has been used to increase the reliability of OSCE in a highstake setting (such as The Pharmacy Examining Board of Canada's Qualifying Examination), 14 a small OSCE station number, in conjunction with the written exam, can serve as a reliable, valid and economically feasible global assessment method of student performance. 18,20 Although we did not conduct any interrater reliability testing, the similarity in the exam scores indicated that the interrater and interpatient variability didn't affect reliability of the exam.…”

Competencia clínica de los médicos de familia en 14 Comunidades Autónomas españolas

“…Diferentes estudios en los que se evalúa la competencia clínica se han realizado en poblaciones de médicos residentes en formación 20,21 , o de estudiantes de medicina 22,23 , y si bien es cierto que también se dispone de datos publicados en poblaciones de médicos en ejercicio 10,17 , son poblaciones menores y normalmente referidos a una CCAA concreta. La mayor experiencia en la evaluación de la competencia clíni-ca de profesionales en ejercicio en España corresponde al Institut d'Estudis de la Salut (IES), que trabaja en la metodología ECOE en España desde hace más de 20 años, y que, por tanto, dispone de la mejor casuística en nuestro país 5 .…”

“Teaching Ophthalmology for Undergraduate Medical Students” 40 Years Experience in Problem Based, Community Oriented Institutions

“…Weekly quiz were put for them to keep connect with the course tutor. OSCE is the standard system for evaluation to all medical students in clinical phase education [9]. Questionnaire was used to monitor student satisfaction and their recommendations to be taken in consideration for future groups.…”