Assessing the Transition of Training in Health Systems Science From Undergraduate to Graduate Medical Education

Abstract

Background The American Medical Association Accelerating Change in Medical Education (AMA-ACE) consortium proposes that medical schools include a new 3-pillar model incorporating health systems science (HSS) and basic and clinical sciences. One of the goals of AMA-ACE was to support HSS curricular innovation to improve residency preparation. Objective This study evaluates the effectiveness of HSS curricula by using a large dataset to link medical school graduates to internship Milestones through collaboration with the Accreditation Council for Graduate Medical Education (ACGME). Methods ACGME subcompetencies related to the schools' HSS curricula were identified for internal medicine, emergency medicine, family medicine, obstetrics and gynecology (OB/GYN), pediatrics, and surgery. Analysis compared Milestone ratings of ACE school graduates to non-ACE graduates at 6 and 12 months using generalized estimating equation models. Results At 6 months both groups demonstrated similar HSS-related levels of Milestone performance on the selected ACGME competencies. At 1 year, ACE graduates in OB/GYN scored minimally higher on 2 systems-based practice (SBP) subcompetencies compared to non-ACE school graduates: SBP01 (1.96 vs 1.82, 95% CI 0.03-0.24) and SBP02 (1.87 vs 1.79, 95% CI 0.01-0.16). In internal medicine, ACE graduates scored minimally higher on 3 HSS-related subcompetencies: SBP01 (2.19 vs 2.05, 95% CI 0.04-0.26), PBLI01 (2.13 vs 2.01; 95% CI 0.01-0.24), and PBLI04 (2.05 vs 1.93; 95% CI 0.03-0.21). For the other specialties examined, there were no significant differences between groups. Conclusions Graduates from schools with training in HSS had similar Milestone ratings for most subcompetencies and very small differences in Milestone ratings for only 5 subcompetencies across 6 specialties at 1 year, compared to graduates from non-ACE schools. These differences are likely not educationally meaningful.

“Finding My Piece in That Puzzle”: A Qualitative Study Exploring How Medical Students at Four U.S. Schools Envision Their Future Professional Identity in Relation to Health Systems

“…26 While some residents excel within particular HSS domains, 27 milestone-based ratings of performance during internship do not differ between graduates of medical schools with and without HSS curricula. 28 This could in part reflect a limited sensitivity of milestone-based assessments 28 and also suggests that factors outside the HSS curriculum may contribute to identification with systems-related roles.…”