Predicting Osteopathic Medical Student Performance on the United States Medical Licensing Examination Step 2 Clinical Knowledge From Results of the Comprehensive Osteopathic Medical Licensing Examination Level 2-Cognitive Evaluation

Background Many residency programs do not accept the Comprehensive Osteopathic Medical Licensing Examination (COMLEX-USA) alone for osteopathic applicants. Furthermore, among those programs that do accept the COMLEX-USA, it is unknown how programs scale their minimum COMLEX-USA scores compared to their minimum United States Medical Licensing Examination (USMLE) scores. Objective Our objective was to examine the variation of relative within-program differences between minimum USMLE Step and COMLEX-USA Level scores required for consideration by United States residency programs. Methods We performed a cross-sectional analysis of the Fellowship and Residency Electronic Interactive Database Access (FREIDA) database from April 2023, including the 10 specialties with the most training spots in 2022. These specialties were internal medicine, family medicine, pediatrics, emergency medicine, psychiatry, surgery, anesthesiology, obstetrics-gynecology, orthopedic surgery, and neurology. Within-program differences were calculated by subtracting the minimum USMLE Step 1 and 2 scores from the converted minimum USMLE Step 1 and 2 scores calculated from the minimum COMLEX-USA Level 1 and 2 scores using two conversion tools. We present differences as medians with interquartile ranges (IQR). Additionally, we report the proportion of programs with greater than 10-point differences for each step (1 and 2). Results Of the 3,364 accredited programs from the examined specialties, we included 1,477 in the Step 1 analysis and 1,227 in the Step 2 analysis with complete data. The median within-program difference between the minimum Step 1 score and the predicted Step 1 score was 12.0 (IQR 2.0, 17.0) using the Barnum and colleagues’ conversion tool and -1.7 (IQR -6.2, 6.3) using the Smith and colleagues’ tool. The median differences for Step 2 were 2.0 (IQR -8.0, 12.0) and -6.5 (IQR -13.9, -1.5) for each tool, respectively. Using the Barnum and Smith conversion tools, 937 (63%) and 435 (29%) programs had a greater than 10-point Step 1 score difference, respectively. Similarly, for Step 2, 564 (46%) and 515 (42%) programs had a greater than 10-point difference with each conversion tool. Conclusion There is wide variation in the within-program differences between minimum USMLE and predicted minimum USMLE (from COMLEX-USA) scores. Many programs have greater than 10-point differences, which may be a source of bias in osteopathic applicant selection.

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Minimum United States Medical Licensing Examination and Comprehensive Osteopathic Medical Licensing Examination Scores Often Do Not Align

Nikolla,

Mudrakola,

Feronti

et al. 2023

“…There are multiple studies in the literature that have attempted to calculate a score convertor for both USMLE step 1 and step 2 clinical knowledge (step 2CK) to allow direct comparison of those without USMLE scores to those who do. These studies have been found to under predict USMLE scores and have not been validated in large population studies [20][21][22]. Additionally, the examinations themselves have differing goals and content, essentially forcing osteopathic medical students interested in competitive specialties to take the USMLE [23].…”

Section: Schlitzkus Et Al (2012) Highlight Their Study Comparing the ...mentioning

confidence: 99%

Identifying Attitudes Toward and Acceptance of Osteopathic Graduates in Surgical Residency Programs in the Era of Single Accreditation: Results of the American College of Osteopathic Surgeons Medical Student Section Questionnaire of Program Directors

Heard¹,

Buckley²,

Burns³

et al. 2022

“…For example, Step 1 scores have been found to correlate directly with the percentage of questions answered correctly on UWorld and with NBME CBSE scores [ 5 ]. Higher scores on Step 1 have also demonstrated predictive value for high scores on NBME subject exams, with one study showing the use of Step 1 scores to predict subject exam scores within 0.2 points [ 6 ]. This relationship between Step 1 scores and subject exam scores was useful, however the recent shift to a pass/fail rubric means this needs to be reexamined.…”

Section: Introductionmentioning

confidence: 99%

The Early Impact of Deciding to Take the United States Medical Licensing Examination Step 1 for Osteopathic Medical Students in the Pass/Fail Era

Hedgepeth¹,

Wlasowicz²,

Lott³

et al. 2023

Self Cite

Introduction: The United States Medical Licensing Examination (USMLE)Step 1 recently shifted from a three-digit scoring format to a pass/fail scoring rubric. Lake Erie College of Osteopathic Medicine (LECOM) is among a number of osteopathic medical schools that traditionally included passing Step 1 as a graduation requirement. However, LECOM removed this requirement following the change in scoring format. National Board of Medical Examiners (NBME) subject examinations have a significant impact on the clerkship grades of third-year medical students. Therefore, our pilot study compared NBME subject examination scores among third-year LECOM medical students that did and did not take and pass Step 1. We anticipate that both high pre-clinical grade point average (GPA) and having passed Step 1 will be associated with higher subject exam scores, but passing Step 1 will have a relationship with higher subject exam scores that is independent of pre-clinical GPA.Methods: Using voluntary response sampling, 201 osteopathic medical students from LECOM completed an online survey through Google Forms regarding their pre-clinical GPA, subject exam scores, whether they took and passed USMLE Step 1, and their study resources used throughout clerkships.Results: There was a positive correlation (p < 0.05) found between pre-clinical GPA and exam scores across all subjects among students that took Step 1. There was no relationship between pre-clinical GPA and exam scores across all subjects among students that did not take Step 1 (p > 0.05). Students that took Step 1 had a higher pre-clinical GPA than those that did not. Students that took and passed Step 1 scored higher on subject exams. Fifty-nine percent of respondents indicated they would have studied more for Step 1 if these exams were scored on the three-digit format, while zero respondents indicated they would have studied less.Conclusion: Although higher pre-clinical GPA and taking Step 1 were associated with higher scores on subject exams, taking Step 1 appears to have an independent influence on subject exams because there was no relationship found between pre-clinical GPA and subject exam scores among students that did not take Step 1. Therefore, there may be features related to preparing for this exam that better equip osteopathic medical students to perform well on subject exams.