Virtual Surgery Oral Board Examinations in the Era of COVID-19 Pandemic. How I Do It!

Shebrain, Saad; Nava, Kristofer E.; Munene, Gitonga; Shattuck, Cynthia; Collins, John T.; Sawyer, Robert G.

doi:10.1016/j.jsurg.2020.09.012

Cited by 13 publications

(9 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Two multi-institutional studies focused on vascular and cardiothoracic surgery trainees, for example, found the virtual mock oral platform to be both feasible and useful for participants [ 17 , 18 ]. Similar pilot experiences have been published in the general surgery literature indicating that both faculty and trainee participants are satisfied with the experience and seek standardization of the process moving forward [ 19 , 20 ]. In these pilot studies, trainees were primarily evaluated with pass–fail case rubrics and written feedback.…”

Section: Introductionmentioning

confidence: 65%

The Advanced Certifying Exam Simulation-Pro assessment instrument: evaluating surgical trainee examsmanship in virtual oral exams

et al. 2023

View full text Add to dashboard Cite

Purpose In response to the COVID-19 pandemic, many educational activities in general surgery residency have shifted to a virtual environment, including the American Board of Surgery (ABS) Certifying Exam. Virtual exams may become the new standard. In response, we developed an evaluation instrument, the ACES-Pro, to assess surgical trainee performance with a focus on examsmanship in virtual oral board examinations. The purpose of this study was two-fold: (1) to assess the utility and validity of the evaluation instrument, and (2) to characterize the unique components of strong examsmanship in the virtual setting, which has distinct challenges when compared to in-person examsmanship. Methods We developed a 15-question evaluation instrument, the ACES-Pro, to assess oral board performance in the virtual environment. Nine attending surgeons viewed four pre-recorded oral board exam scenarios and scored examinees using this instrument. Evaluations were compared to assess for inter-rater reliability. Faculty were also surveyed about their experience using the instrument. Results Pilot evaluators found the ACES-Pro instrument easy to use and felt it appropriately captured key professionalism metrics of oral board exam performance. We found acceptable inter-rater reliability in the domains of verbal communication, non-verbal communication, and effective use of technology (Guttmann’s lambda-2 were 0.796, 0.916, and 0.739, respectively). Conclusions The ACES-Pro instrument is an assessment with evidence for validity as understood by Kane’s framework to evaluate multiple examsmanship domains in the virtual exam setting. Examinees must consider best practices for virtual examsmanship to perform well in this environment. Supplementary Information The online version contains supplementary material available at 10.1007/s44186-023-00107-7.

show abstract

Section: Introductionmentioning

confidence: 65%

The Advanced Certifying Exam Simulation-Pro assessment instrument: evaluating surgical trainee examsmanship in virtual oral exams

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Our successes have duplicated those with the general surgery VCE across all the various components surveyed and were consistent with the results reported for the two mock oral VCEs for vascular and general surgery trainees. 1 , 4 , 5 The incorporation of the images required both establishing a secure method for sharing the images and a mechanism by which the candidates could request additional images not provided in the initial stem. This required developing a strategy such that they did not know whether any additional images were available for each question.…”

Section: Discussionmentioning

confidence: 99%

Implementation of the Vascular Surgery Board virtual certifying examination

Huber

Brown

Lee

et al. 2022

Journal of Vascular Surgery

View full text Add to dashboard Cite

“…There is evidence that online assessment is a valid tool for assessing knowledge, and that it holds benefit to both the examiner and the candidate (13,15). The challenge lies in its implementation.…”

Section: Discussionmentioning

confidence: 99%

Use Of Online Structured Oral Examinations For Postgraduate Specialist And Subspecialist Certification Examinations During The COVID-19 Pandemic In South Africa

Burch

McGuire

Mbouaffou

et al. 2022

Preprint

View full text Add to dashboard Cite

Introduction: The COVID-19 pandemic prevented many medical specialist programmes across the world from conducting certification examinations. However, the Colleges of Medicine of South Africa (CMSA), which is responsible for conducting all specialist and sub-specialist certification examinations in South Africa, continued conducting examinations despite the pandemic.Objectives: The study was conducted to determine the success and challenges of conducting SOEs using Zoom, with a special focus on acceptability of this examination format to candidates.Method: The CMSA hosted 2933 candidates undertaking examinations for 95 diploma, specialist and sub-specialist qualifications. Of these, 2176 candidates participated in examinations requiring SOEs which amounted to 7226 individual Zoom calls. A survey was conducted during the first cycle of examinations in 2020 to assess the acceptability of the examination format to candidates.Results: Pass rates ranged from 82% to 96% for the respective qualifications across two examination cycles. The overall pass rate for these 2 examination cycles was higher than that of the previous 6 examination cycles for the CMSA. The majority of 542 candidates who completed the survey felt that it was a fair examination method that tested their clinical reasoning and insight appropriately. They also felt that the case scenarios were of adequate complexity, and that the material was well presented. Conclusion: This ambitious undertaking has demonstrated that virtual, case-based SOEs are a feasible and acceptable examination option. It is anticipated that the positive outcome of this study, the significant time and cost savings to candidates, examiners and the organization will motivate member colleges to support the ongoing use of virtual SOEs as part of the composite examination package.

show abstract

Virtual Surgery Oral Board Examinations in the Era of COVID-19 Pandemic. How I Do It!

Cited by 13 publications

References 6 publications

The Advanced Certifying Exam Simulation-Pro assessment instrument: evaluating surgical trainee examsmanship in virtual oral exams

The Advanced Certifying Exam Simulation-Pro assessment instrument: evaluating surgical trainee examsmanship in virtual oral exams

Implementation of the Vascular Surgery Board virtual certifying examination

Use Of Online Structured Oral Examinations For Postgraduate Specialist And Subspecialist Certification Examinations During The COVID-19 Pandemic In South Africa

Contact Info

Product

Resources

About