Association of Surgical Resident Competency Ratings With Patient Outcomes

Kendrick, Daniel E.; Thelen, Angela E.; Chen, Xilin; Gupta, Tanvi; Yamazaki, Kenji; Krumm, Andrew E.; Bandeh-Ahmadi, Hoda; Clark, M. R.; Luckoscki, John; Fan, Zhining; Wnuk, Greg M.; Ryan, Andrew M.; Mukherjee, Bhramar; Hamstra, Stanley J.; Dimick, Justin B.; Holmboe, Eric S.; George, Brian C.

doi:10.1097/acm.0000000000005157

Cited by 14 publications

(28 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although this evidence is promising and conceptually consistent with earlier findings linking medical school performance with disciplinary action by state medical boards, further investigations are warranted . For example, another recent research report by Kendrick and colleagues failed to demonstrate an association between ACGME Milestone competency ratings of US general surgical residents and those surgeons’ Medicare claims in early career practice.…”

supporting

confidence: 78%

Competency-Based Medical Education—A Journey or a Destination?

Grover

Howley

2023

JAMA Netw Open

View full text Add to dashboard Cite

supporting

confidence: 78%

Competency-Based Medical Education—A Journey or a Destination?

Grover

Howley

2023

JAMA Netw Open

View full text Add to dashboard Cite

“…The authors report no association between mean milestone ratings of general surgery resident graduates and their early-career patient outcomes, including on subanalyses of milestone ratings related to both technical skill and operative performance. 7 With increasing national pressure on graduate medical education (GME) programs, many education leaders have called for using large national datasets to evaluate associations between educational and clinical outcomes data to provide evidence of GME effectiveness and identify improvement opportunities. [8][9][10] We applaud the collaborative efforts of the author group and their development of this novel linkage between surgical resident Milestone ratings and early-career surgeon patient outcomes.…”

Section: Milestone Ratings and Patient Outcomesmentioning

confidence: 99%

Section: Milestone Ratings and Patient Outcomesmentioning

confidence: 99%

“…In this issue of Academic Medicine , Kendrick et al 7 report findings from analysis of a novel data linkage between ACGME milestone ratings for early-career, non-fellowship-trained general surgery graduates and their risk-adjusted complication and mortality rates for high-risk, common inpatient operations based on national Medicare claims data. The authors report no association between mean milestone ratings of general surgery resident graduates and their early-career patient outcomes, including on subanalyses of milestone ratings related to both technical skill and operative performance.…”

Section: Milestone Ratings and Patient Outcomesmentioning

confidence: 99%

“…This analysis was well designed, including risk adjustment of patient outcomes and a study population of only non-fellowship-trained surgeons to avoid the potential confounding of additional postresidency training on surgeon performance. 7 The group also used mean milestone ratings from the midyear time point of graduating residents’ final year (instead of final rating before graduation) to reduce potential bias from CCCs that may feel pressure to artificially inflate graduating residents’ ratings in their final milestone evaluation to a “proficient” level. 7 However, despite these careful parameters to limit confounders and enhance the quality and reliability of the study, no difference was identified in patient outcomes based on assessments of surgeon performance near the end of their training.…”

Section: Milestone Ratings and Patient Outcomesmentioning

confidence: 99%

See 2 more Smart Citations

Using Graduating Surgical Resident Milestone Ratings to Predict Patient Outcomes: A Blunt Instrument for a Complex Problem

Montgomery

Lindeman

2023

Academic Medicine

View full text Add to dashboard Cite

In 2013, U.S. general surgery residency programs implemented a milestones assessment framework in an effort to incorporate more competency-focused evaluation methods. Developed by a group of surgical education leaders and other stakeholders working with the Accreditation Council for Graduate Medical Education and recently updated in a version 2.0, the surgery milestones framework is centered around 6 “core competencies”: patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice. While prior work has focused on the validity of milestones as a measure of resident performance, associations between general surgery resident milestone ratings and their post-training patient outcomes have only recently been explored in an analysis in this issue of Academic Medicine by Kendrick et al. Despite their well-designed efforts to tackle this complex problem, no relationships were identified. This accompanying commentary discusses the broader implications for the use of milestone ratings beyond their intended application, alternative assessment methods, and the challenges of developing predictive assessments in the complex setting of surgical care. Although milestone ratings have not been shown to provide the specificity needed to predict clinical outcomes in the complex settings studied by Kendrick et al, hope remains that utilization of other outcomes, assessment frameworks, and data analytic tools could augment these models and further our progress toward a predictive assessment in surgical education. Evaluation of residents in general surgery residency programs has grown both more sophisticated and complicated in the setting of increasing patient and case complexity, constraints on time, and regulation of resident supervision in the operating room. Over the last decade, surgical education research efforts related to resident assessment have focused on measuring performance through accurate and reproducible methods with evidence for their validity, as well as on attempting to refine decision making about resident preparedness for unsupervised practice.

show abstract

Associations of Internal Medicine Residency Milestone Ratings and Certification Examination Scores With Patient Outcomes

Gray,

Vandergrift,

Stevens

et al. 2024

JAMA

View full text Add to dashboard Cite

ImportanceDespite its importance to medical education and competency assessment for internal medicine trainees, evidence about the relationship between physicians’ milestone residency ratings or the American Board of Internal Medicine’s initial certification examination and their hospitalized patients’ outcomes is sparse.ObjectiveTo examine the association between physicians’ milestone ratings and certification examination scores and hospital outcomes for their patients.Design, Setting, and ParticipantsRetrospective cohort analyses of 6898 hospitalists completing training in 2016 to 2018 and caring for Medicare fee-for-service beneficiaries during hospitalizations in 2017 to 2019 at US hospitals.Main Outcomes and MeasuresPrimary outcome measures included 7-day mortality and readmission rates. Thirty-day mortality and readmission rates, length of stay, and subspecialist consultation frequency were also assessed. Analyses accounted for hospital fixed effects and adjusted for patient characteristics, physician years of experience, and year.ExposuresCertification examination score quartile and milestone ratings, including an overall core competency rating measure equaling the mean of the end of residency milestone subcompetency ratings categorized as low, medium, or high, and a knowledge core competency measure categorized similarly.ResultsAmong 455 120 hospitalizations, median patient age was 79 years (IQR, 73-86 years), 56.5% of patients were female, 1.9% were Asian, 9.8% were Black, 4.6% were Hispanic, and 81.9% were White. The 7-day mortality and readmission rates were 3.5% (95% CI, 3.4%-3.6%) and 5.6% (95% CI, 5.5%-5.6%), respectively, and were 8.8% (95% CI, 8.7%-8.9%) and 16.6% (95% CI, 16.5%-16.7%) for mortality and readmission at 30 days. Mean length of stay and number of specialty consultations were 3.6 days (95% CI, 3.6-3.6 days) and 1.01 (95% CI, 1.00-1.03), respectively. A high vs low overall or knowledge milestone core competency rating was associated with none of the outcome measures assessed. For example, a high vs low overall core competency rating was associated with a nonsignificant 2.7% increase in 7-day mortality rates (95% CI, −5.2% to 10.6%; P = .51). In contrast, top vs bottom examination score quartile was associated with a significant 8.0% reduction in 7-day mortality rates (95% CI, −13.0% to −3.1%; P = .002) and a 9.3% reduction in 7-day readmission rates (95% CI, −13.0% to −5.7%; P &lt; .001). For 30-day mortality, this association was −3.5% (95% CI, −6.7% to −0.4%; P = .03). Top vs bottom examination score quartile was associated with 2.4% more consultations (95% CI, 0.8%-3.9%; P &lt; .003) but was not associated with length of stay or 30-day readmission rates.Conclusions and RelevanceAmong newly trained hospitalists, certification examination score, but not residency milestone ratings, was associated with improved outcomes among hospitalized Medicare beneficiaries.

show abstract

Association of Surgical Resident Competency Ratings With Patient Outcomes

Cited by 14 publications

References 47 publications

Competency-Based Medical Education—A Journey or a Destination?

Competency-Based Medical Education—A Journey or a Destination?

Using Graduating Surgical Resident Milestone Ratings to Predict Patient Outcomes: A Blunt Instrument for a Complex Problem

Associations of Internal Medicine Residency Milestone Ratings and Certification Examination Scores With Patient Outcomes

Contact Info

Product

Resources

About