Assessment methods and the validity and reliability of measurement tools in online objective structured clinical examinations: a systematic scoping review

Abstract

The coronavirus disease 2019 (COVID-19) pandemic has required educators to adapt the in-person objective structured clinical examination (OSCE) to online settings in order for it to remain a critical component of the multifaceted assessment of a student’s competency. This systematic scoping review aimed to summarise the assessment methods and validity and reliability of the measurement tools used in current online OSCE (hereafter, referred to as teleOSCE) approaches. A comprehensive literature review was undertaken following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidelines. Articles were eligible if they reported any form of performance assessment, in any field of healthcare, delivered in an online format. Two reviewers independently screened the results and analysed relevant studies. Eleven articles were included in the analysis. Pre-recorded videos were used in 3 studies, while observations by remote examiners through an online platform were used in 7 studies. Acceptability as perceived by students was reported in 2 studies. This systematic scoping review identified several insights garnered from implementing teleOSCEs, the components transferable from telemedicine, and the need for systemic research to establish the ideal teleOSCE framework. TeleOSCEs may be able to improve the accessibility and reproducibility of clinical assessments and equip students with the requisite skills to effectively practice telemedicine in the future.

Evaluation of the Utility of Online Objective Structured Clinical Examination Conducted During the COVID-19 Pandemic

“…We think that an added positive educational impact of this experience is training students on using technology in learning and, later after graduation, in health care. This is supported by the findings of Felthun et al 16 in their systematic scoping review as they reported that teleOSCE could be able to equip the students with the requisite skills of practicing telemedicine in the future.…”

“…Similar results were reported by Shaban et al, 13 who reported comparable Cronbach’s alpha values between online and conventional OSCE at their institution. Furthermore, Felthun et al 16 reported in their systematic scoping review that teleOSCE (online OSCE) could be able to improve the reproducibility of clinical assessments.…”

Empirical analysis comparing the tele-objective structured clinical examination and the in-person assessment in Australia

“…Recent efforts to deliver the teleOSCE and online OSCE orientation have been acceptable to students and examiners respectively [8,9]. A systematic review summarised the various attempts in the literature to incorporate an online element and discovered that while there is generally good validity and reliability, there is a need for systemic research to guide the ideal teleOSCE format [10].…”

“…Recent efforts to deliver the tele-objective structured clinical examination (teleOSCE) and online OSCE orientation have been acceptable to students and examiners respectively [ 8 , 9 ]. A systematic review summarized the various attempts in the literature to incorporate an online element and discovered that while there is generally good validity and reliability, there is a need for systemic research to guide the ideal teleOSCE format [ 10 ]. As yet, no study has attempted to define and discuss the impact on the assessment outcome of separating students, patients, and assessors into different locations, and executing examinations online.…”

Applying the Student Response System in the Online Dermatologic Video Curriculum on Medical Students' Interaction and Learning Outcomes during the COVID-19 Pandemic

“…Online lectures in medical education have become prevalent with the advancement of technology and play important roles. [ 1 2 3 4 5 ] It offers students convenience and flexibility in terms of scheduling, circumstances, and transportation. [ 6 ] However, learning online is challenging.…”