Sustained Clinical Performance During Surgical Rotations Predicts NBME Shelf Exam Outcomes

Tracy, Brett M.; Hazen, Benjamin; Ward, Collin P.; Winer, Joshua H.; Pettitt, Barbara J.

doi:10.1016/j.jsurg.2020.06.033

Cited by 5 publications

(3 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…21 Scores on the Surgery Subject Examination have been found to correlate with OSCE scores, oral exam scores, evaluations of clinical performance, and more clinical experience. 20,[22][23][24][25][26] However, the Surgery Subject Examination is not a direct measure of clinical performance. The current results shed light on the effect of clerkship length reductions on clinical knowledge, not clinical performance.…”

Section: Discussionmentioning

confidence: 99%

Clerkship length: Does it affect performance on the NBME Surgery Subject Examination across schools?

Wynne

O’Donnell

Morrison

et al. 2022

Preprint

View full text Add to dashboard Cite

Background: The optimal duration for clerkships has not yet been established. The purpose of this study was to add to the limited literature on the impact of clerkship length using data from multiple medical schools and controlling for differences in pre-clerkship achievement. Methods: A two-way analysis of covariance (ANCOVA) was conducted to examine the effects of clerkship length reductions and school affiliation on students’ performance on the Subject Examination in Surgery. Results: The relationship between surgery clerkship length and school performance while controlling for Step 1 scores, F (4, 1708) = 2.69, p = .030, partial η2 = .006, indicating that the effect of a clerkship length reduction on Surgery Subject Examination performance was not consistent. Further, across 91 medical schools, clerkship length was not a significant predictor of performance. Conclusion: This study suggests that it is possible to reduce the duration of a surgery clerkship without negatively impacting student achievement.

show abstract

Section: Discussionmentioning

confidence: 99%

Clerkship length: Does it affect performance on the NBME Surgery Subject Examination across schools?

Wynne

O’Donnell

Morrison

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The National Board of Medical Examiners (NBME) clinical subject exams serve as an important measure of a medical student's knowledge and clinical capabilities [ 11 - 12 ]. The NBME subject exams are taken by medical students, typically at the conclusion of a clinical rotation in the respective field.…”

Section: Introductionmentioning

confidence: 99%

Comparing the Performance of Popular Large Language Models on the National Board of Medical Examiners Sample Questions

Abbas,

Rehman,

Rehman

2024

Cureus

View full text Add to dashboard Cite

Introduction: Large language models (LLMs) have transformed various domains in medicine, aiding in complex tasks and clinical decision-making, with OpenAI's GPT-4, GPT-3.5, Google’s Bard, and Anthropic’s Claude among the most widely used. While GPT-4 has demonstrated superior performance in some studies, comprehensive comparisons among these models remain limited. Recognizing the significance of the National Board of Medical Examiners (NBME) exams in assessing the clinical knowledge of medical students, this study aims to compare the accuracy of popular LLMs on NBME clinical subject exam sample questions. Methods: The questions used in this study were multiple-choice questions obtained from the official NBME website and are publicly available. Questions from the NBME subject exams in medicine, pediatrics, obstetrics and gynecology, clinical neurology, ambulatory care, family medicine, psychiatry, and surgery were used to query each LLM. The responses from GPT-4, GPT-3.5, Claude, and Bard were collected in October 2023. The response by each LLM was compared to the answer provided by the NBME and checked for accuracy. Statistical analysis was performed using one-way analysis of variance (ANOVA). Results: A total of 163 questions were queried by each LLM. GPT-4 scored 163/163 (100%), GPT-3.5 scored 134/163 (82.2%), Bard scored 123/163 (75.5%), and Claude scored 138/163 (84.7%). The total performance of GPT-4 was statistically superior to that of GPT-3.5, Claude, and Bard by 17.8%, 15.3%, and 24.5%, respectively. The total performance of GPT-3.5, Claude, and Bard was not significantly different. GPT-4 significantly outperformed Bard in specific subjects, including medicine, pediatrics, family medicine, and ambulatory care, and GPT-3.5 in ambulatory care and family medicine. Across all LLMs, the surgery exam had the highest average score (18.25/20), while the family medicine exam had the lowest average score (3.75/5). Conclusion: GPT-4's superior performance on NBME clinical subject exam sample questions underscores its potential in medical education and practice. While LLMs exhibit promise, discernment in their application is crucial, considering occasional inaccuracies. As technological advancements continue, regular reassessments and refinements are imperative to maintain their reliability and relevance in medicine.

show abstract

“…Surgery clerkship for medical students has been changed in response to clinical exposure limitation due to this pandemic to reach a level of competence in knowledge, hands-on, and clinical experience [ 2 ]. Whereas the student's clinical performance in surgery clerkship related to surgical knowledge, clinical competence, and reasoning [ 3 ]. Suspended rotation may implicate clinical competence, and a virtual platform may be an alternative to delivering surgery competence and experience [ 4 ].…”

Section: Introductionmentioning

confidence: 99%

Evaluating knowledge and skill in surgery clerkship during covid 19 pandemics: A single-center experience in Indonesia

Setiawan

Sugeng

Luailiyah

et al. 2021

Annals of Medicine &Amp; Surgery

View full text Add to dashboard Cite

Backgrounds Surgery clerkship for medical students has been changed in response to clinical exposure limitation due to this pandemic. This study aim to evaluate knowledge and skill of students in surgery clerkship in covid 19 pandemics. Methods Cross-Sectional design comparing surgery clerkship before and during COVID-19.A total of 270 fourth and fifth-year medical students have enrolled in surgery clerkship from June 2019–October 2020 were selected for this study. Each student had completed education and training in the hospital for nine weeks in the rotation. Results There is no significant difference in MCQs scores before and during the pandemic. However, a significant difference was found in OSCE scores. Conclusions Combining virtual platforms and in-person clinical rotation is an effective surgery clerkship curriculum, particularly in pandemic covid 19. There are no different skill and knowledge results before and during the pandemic analyzed from MCQs and OSCE exam.

show abstract

Sustained Clinical Performance During Surgical Rotations Predicts NBME Shelf Exam Outcomes

Cited by 5 publications

References 21 publications

Clerkship length: Does it affect performance on the NBME Surgery Subject Examination across schools?

Clerkship length: Does it affect performance on the NBME Surgery Subject Examination across schools?

Comparing the Performance of Popular Large Language Models on the National Board of Medical Examiners Sample Questions

Evaluating knowledge and skill in surgery clerkship during covid 19 pandemics: A single-center experience in Indonesia

Contact Info

Product

Resources

About