Innovative Method to Digitize a Web-Based OSCE Evaluation System for Medical Students: A Cross-Sectional Study in University Hospital in Saudi Arabia

Abstract

Purpose The Objective Structured Clinical Examination (OSCE) is a standard academic assessment tool in the field of medical education. This study presents an innovative method for digitizing OSCE evaluation system for medical students and explores its efficacy compared to the traditional paper-based system, through the analysis of a User Satisfaction Survey. Methods A cross-sectional, questionnaire-based study involving a User Satisfaction Survey to evaluate assessors’ attitudes toward and acceptance of the Computerized Web-based OSCE Evaluation System (COES) was used. Fifth-year medical students at a College of Medicine were assessed clinically through their 2019 end-of-year OSCE examination by 30 examiners in five different OSCE stations. The traditional paper-based stations were converted into an online electronic version using QuestionPro software. Answers were filled in using smart tablets (iPads). QR codes were used for students’ identification at each station to fully digitize the process and save time. After the completion of the exam, a User Satisfaction Survey was sent electronically to all examiners to evaluate their experiences with the new system. Results The response rate for the survey was 100% with an internal consistency of 0.83. Almost all the examiners (29, 97%) were satisfied with the application of the COES. Further, 72% of the examiners indicated that the electronic system facilitated the evaluation of the students’ skills, and 84% found using a smart device (iPad) was easier than using a paper form. All examiners expressed their preference for using the electronic system in the future. Conclusion Users were satisfied with the utilization of the customized COES. This concept of fully digitizing the OSCE assessment process shortened the time needed for both the analysis of results and providing students with feedback. Further observational studies are needed to assess examiners’ behaviors when using this methodology.

Health science students’ perceptions about Objective Structured Clinical Examination (OSCE) as a method of clinical evaluation

“…As a result, the more OSCE assessment audit carried out, the more advanced the evaluation method of OSCE in an institution [31,42]. Hence, many educators consider OSCE as an effective evaluation method for practical clinical skills because of the significant benefits of OSCE over the disadvantages that it causes [31,49,50,51].…”

Pengembangan Sistem Evaluasi OSCE Berbasis Aplikasi Ponsel untuk Mahasiswa Kedokteran di Indonesia

“…Potensi dalam pengembangan aplikasi ini bisa memudahkan evaluasi yang terdata sehingga memudahkan evaluasi kuantitatif berkelanjutan dengan mudah. (Chandran et al, 2022;Chase et al, 2018;Swanwick et al, 2019;Yousef et al, 2022) Med Stud Games (MSG) adalah salah satu aplikasi pendidikan kedokteran popular dikalangan mahasiswa kedokteran di Indonesia. Aplikasi ini digunakan secara individu oleh mahasiswa kedokteran untuk berlatih pengetahuan kedokteran sebelum ujian.…”

“…Tidak hanya akan menghemat penggunaan kertas, tapi kualitas evaluasi dapat diperkaya secara praktis dalam kegiatan pembelajaran dan latihan OSCE. (Yousef et al, 2022) Pada penerapan lain seperti pembelajaran online yang menjadi popular setelah pandemi COVID-19, metode evaluasi OSCE campur / Blended learning juga diterapkan di berbagai fakultas kedokteran. Hal ini juga terjadi pada sebagian responden penelitian ini.…”

Performance evaluation of standardized residency training based on non-parametric intelligent group decision method

“…Originating from Dundee University, the OSCE serves as a standard tool in medical education to evaluate practical and communicative proficiencies of medical students. While it bolsters examinee confidence and gauges clinical acumen across diverse settings, its drawbacks include substantial time investment and the potential for individual documentation errors [7] . Primarily utilized for formative assessment, the 360degree evaluation offers comprehensive insight into residents' competence from various angles, albeit influenced by examiner subjectivity [8] .…”