The Impact of Postgraduate Training and Timing on USMLE Step 3 Performance

Abstract

Study data reaffirm the validity of the USMLE Step 3 examination, and the information found in the pattern of results across specialties may be useful to residents and program directors.

Does Step 3 of the United States Medical Licensing Exam Measure Clinical Competence? A Predictive Validity Study

“…It may be that medical school performance (e.g., years 1, 2, and 3) correlations with Step 3 are curriculum specific. Nonetheless, our results and interpretation are consistent with the findings of others [5,6,9]. Future work should include other institutions and larger samples which may also be profitably studied longitudinally, beyond residency.…”

“…Additionally, when they used Step 2 scores as a covariate, the performance of examinees in the narrowly focused postgraduate training programs tended to decrease as training progressed while for the broadly focused programs, performance increased. These patterns of results provide evidence for the validity of Step 3 as a measure of clinical skills [5]. In a more recent sample (n = 40,588 between 2007 and 2009), Feinberg et al [6] found a similar pattern of results on Step 3 for broadly focused (variety of patients under a variety of conditions) versus narrowly focused (limited exposure to a variety of patients and conditions) residencies.…”

“…A few studies have addressed this question, albeit circuitously. In a large-scale study of 36,805 medical school graduates who took Step 3 between 1999 and 2002, for example, Sawhill and colleagues [5] found that examinees who received postgraduate training that provides exposure to a broad range of patient problems (e.g., emergency medicine, internal medicine) did better on Step 3 than those that had a narrower range (e.g., orthopedics, psychiatry, radiology). Additionally, when they used Step 2 scores as a covariate, the performance of examinees in the narrowly focused postgraduate training programs tended to decrease as training progressed while for the broadly focused programs, performance increased.…”

The Impact of Postgraduate Training on USMLE® Step 3® and its Computer-Based Case Simulation Component

“…Previous research indicates that Step 3 scores are related to the type and length of postgraduate training; specifically, examinees in residency programs who encounter a variety of patients under a variety of conditions (broadly focused programs) obtain higher scores than those in programs with more limited exposure to a variety of patients and conditions (narrowly focused programs), and that those in broadly focused residencies show score increases when Step 3 is taken later in residency training. [2][3][4] …”

Relationship of Admissions Variables and College of Osteopathic Medicine Variables to Performance on COMLEX-USA Level 3

“…[15][16][17][18][19] Multiple studies on the United States Medical Licensing Examination (USMLE) Step 3, the analogous examination used by MD degree-granting medical schools, [20][21][22] have found that performance on USMLE Step 3 depended in part on whether graduates sought a broad-based residency or one with a more narrow focus. [23][24][25] However, no studies could be identified that correlated COMLEX-USA Level 3 performance to admissions variables or performance in the COM curriculum other than to scores on Levels 1 and 2. If performance on Level 3 is to be used in accreditation decisions about COMs, and if admissions variables are at some point to be regarded as relating to Level 3 score, then these relationships need to be elucidated.…”