Results of a National Survey of Program Directors’ Perspectives on a Pass/Fail US Medical Licensing Examination Step 1

Patel, Om U; Haynes, William K; Burge, Kaitlin; Yadav, Ishant; Peterson, Todd David; Camino, Andres F; Wagoner, Nicholas J. Van

doi:10.1001/jamanetworkopen.2022.19212

Cited by 12 publications

(21 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The current study is part of a larger, parent study examining the perspectives of allopathic residency program directors of 25 different specialties. 10 A 14-item survey was developed using Qualtrics and distributed via email. The anonymous survey consisted of five demographic questions, seven questions on the importance of USMLE Step 1 and Step 2 Clinical Knowledge (CK), and two questions asking participants to rank the importance of 16 different applicant metrics.…”

Section: Methodsmentioning

confidence: 99%

Ophthalmology Residency Program Director Survey on Pass/Fail U.S. Medical Licensing Exam Step 1 Scoring

Fujihashi,

Patel,

Yadav

et al. 2023

Journal of Academic Ophthalmology

Self Cite

View full text Add to dashboard Cite

Background Beginning January 26, 2022, the U.S. Medical Licensing Exam (USMLE) Step 1 changed from a numerical score to pass/fail (P/F). The purpose of this study was to determine the perspective of ophthalmology program directors regarding this change in evaluating applicants. Methods After institutional review board approval, a survey was sent out to program directors of all 125 ophthalmology programs accredited by the Accreditation Council for Graduate Medical Education. Survey questions asked for program demographics, the utility of USMLE Step 1 and 2 Clinical Knowledge scores in assessing applicants, and the importance of 16 different applicant metrics before and after Step 1 becomes P/F. The metrics examined were: letters of recommendation; clerkship grades; class ranking; Alpha Omega Alpha Membership; Gold Humanism Honor Society Membership; Dean's Letter; involvement and leadership; personal statement; number of abstracts, presentations, and publications; mean number of research experiences in the specialty; Step 2 Clinical Knowledge score; volunteering; preclinical grades; away rotation in the specialty; the applicant having another graduate degree; and graduation from a top 40 National Institutes of Health-funded program. Data were analyzed using nonoverlapping 95% confidence intervals. Results The survey was completed by 50 (40%) program directors. Sixty-eight percent of respondents stated a student's ranking would be considered more after USMLE Step 1 scores become P/F, and 60% stated medical schools should share clerkship shelf exam scores with residency programs. There were no significant differences in program directors' rankings of applicant metrics following the transition to P/F Step 1. Conclusion Based on our data, program directors will likely not place a greater emphasis on Step 2 scores, despite it being the only remaining objective measure for all applicants following the switch to a P/F Step 1. Nevertheless, program directors expressed an interest in receiving other objective measures, such as shelf exam scores and class ranking, as part of the application process. Notably, we found no significant changes in the rankings of various applicant metrics before and after the transition to P/F Step 1, indicating that the metrics that were important to program directors prior to the change remain just as critical in the new era of admissions.

show abstract

Section: Methodsmentioning

confidence: 99%

Ophthalmology Residency Program Director Survey on Pass/Fail U.S. Medical Licensing Exam Step 1 Scoring

Fujihashi,

Patel,

Yadav

et al. 2023

Journal of Academic Ophthalmology

Self Cite

View full text Add to dashboard Cite

show abstract

“…The search retrieved 493 potential studies (PubMed = 78, Scopus = 331, Web of Science = 81, and PSYCInfo = 2, manual search = 1). After the initial search, 351 duplicates were removed and 44 were excluded after title and abstract review, 4,[7][8][9][10][11][12][13][14][15] based on inclusion and exclusion criteria. [16][17][18][19][20][21][22][23][24][25] Full-text review was performed on 98 articles, [26][27][28][29][30][31][32][33][34][35] of which 45 studies were included [36][37][38][39][40][41][42][43][44][45][46][47][48][49][50] (see Figure, Supplemental Digital Content 5, which illustrates a flow-chart of our literature search, as well as our full search strategies).…”

Section: Study Selectionmentioning

confidence: 99%

Perceptions and Expectations Regarding the Step 1 Score Change Among Surgical and Non-Surgical Program Directors: A Systematic Review and Meta-Analysis

Srinivasan,

Samaan,

Premkumar

et al. 2024

The American Surgeon™

View full text Add to dashboard Cite

There are currently no studies examining differences in perceptions and expected impact of the Step 1 score change to pass/fail between surgical and non-surgical program directors (PDs). We conducted a systematic review in May 2023 of PubMed, Scopus, Web of Science, and PSYCInfo to evaluate studies examining PDs’ perspectives regarding the Step 1 score change. We performed random-effects meta-analyses to determine differences in perspectives among surgical and non-surgical PDs. Surgical PDs (76.8% [95% CI, 72.1%-82.0%], I2 = 52%) reported significantly greater rates of disagreement with the score change compared to non-surgical (65.1% [95% CI, 57.9%-73.1%], I2 = 69.7%) ( P = .01). Surgical PDs also reported significantly greater rates of agreement that the score change will increase the difficulty in objectively comparing applicants (88.1% [95% CI, 84.6%-91.7%], I2 = 16.4%), compared to non-surgical (81.0% [95% CI, 75.6%-86.8%], I2 = 72.6%) ( P = .04). There was less heterogeneity among non-surgical PDs (88.7% [95% CI, 86.2%-91.2%], I2 = 0%), compared to surgical (84.7% [95% CI, 79.0%-90.8%], I2 = 67.3%), regarding expected increases in emphasis on Step 2, although the difference in rates of agreement was not statistically significant. Overall, there is significant heterogeneity in the literature regarding expected changes in the residency application review process. Most PDs reported significant disagreement with the score change, greater expected difficulty in objectively evaluating applicants, and greater emphasis on Step 2, with surgical PDs reporting greater rates of disagreement, greater expected difficulty, and heterogeneity regarding expected increases in emphasis on Step 2, compared to non-surgical. Additionally, there is significant heterogeneity in the overall literature regarding expected changes in the residency application review process. Further research is needed to establish evidence-based guidelines that improve the overall residency application process for all stakeholders.

show abstract

“…2 The transition of the United States Medical Licensing Examination (USMLE) Step 1 and many medical student clerkships to pass/fail grading has led program directors to further rely on just a few attributes and metrics during application review, such as medical school rank and USMLE Step 2 Clinical Knowledge score; this practice may perpetuate inequities in the selection process. 3 Intersectional issues of racism, inequality, and the COVID-19 pandemic 4 have heightened urgency to realize more widespread holistic review, which may improve diversity in resident selection. 5 Yet, a major bottleneck remains.…”

Section: Problemmentioning

confidence: 99%

A New Tool for Holistic Residency Application Review: Using Natural Language Processing of Applicant Experiences to Predict Interview Invitation

Mahtani¹,

Reinstein²,

Marin³

et al. 2023

Academic Medicine

View full text Add to dashboard Cite

show abstract

Results of a National Survey of Program Directors’ Perspectives on a Pass/Fail US Medical Licensing Examination Step 1

Cited by 12 publications

References 3 publications

Ophthalmology Residency Program Director Survey on Pass/Fail U.S. Medical Licensing Exam Step 1 Scoring

Ophthalmology Residency Program Director Survey on Pass/Fail U.S. Medical Licensing Exam Step 1 Scoring

Perceptions and Expectations Regarding the Step 1 Score Change Among Surgical and Non-Surgical Program Directors: A Systematic Review and Meta-Analysis

A New Tool for Holistic Residency Application Review: Using Natural Language Processing of Applicant Experiences to Predict Interview Invitation

Contact Info

Product

Resources

About