The Emergency Medicine Milestones: A Validation Study

Abstract

Objectives: The Accreditation Council for Graduate Medical Education (ACGME) and the American Board of Medical Specialties sought to define milestones for skill and knowledge acquisition during residency training. Milestones are significant objective observable events. The milestones are listed within a structure that is derived from the ACGME general competencies. Major groups of milestones are called "subcompetencies." The original 24 subcompetencies containing 255 milestones for emergency medicine (EM) were developed through a multiorganizational group representing most EM stakeholder groups. To assure that the milestones reflected EM resident progress throughout training, the EM Milestones Working Group (EM MWG) sought to validate the individual milestones.Methods: A computer-based survey was sent to all EM residency programs. The survey period began on April 30, 2012, and concluded on May 15, 2012. Respondents were asked to assign each milestone to a specific level of skill or knowledge acquisition. These levels ranged from a beginning resident to an accomplished clinician. There were two different forms that divided the milestones into two groups of 12 subcompetencies each. Surveys were randomly assigned to programs.Results: There were five respondents (the program director and four key faculty) requested from each of the 159 residences. There were responses from 96 programs (60.4%). Of the 795 survey recipients, 28 were excluded due to prior exposure to the EM milestones. Of the remaining 767 potential respondents, 281 completed the survey (36.6%) within a 16-day period. Based on the survey results, the working group adjusted the milestones in the following ways: one entire subcompetency (teaching) was eliminated, six new milestones were created, 34 milestones were eliminated, 26 milestones were reassigned to a lower level score, and 20 were reassigned to a higher level. Nineteen milestones were edited to provide greater clarity. The final result was 227 discrete milestones among 23 subcompetencies. Conclusions:The EM milestones were validated through a milestone assignment process using a computer-based survey completed by program directors and key faculty. Milestones were revised in accordance with the results to better align assignment within each performance level.ACADEMIC EMERGENCY MEDICINE 2013; 20:730-736 © 2013 by the Society for Academic Emergency Medicine A s part of the Next Accreditation System, the Accreditation Council for Graduate Medical Education (ACGME) has proposed a new method for tracking resident acquisition of knowledge, skills, abilities, attitudes, and experiences.

Assessment methods and resource requirements for milestone reporting by an emergency medicine clinical competency committee

“…In addition, only one FMA was completed per resident so inter-rater agreement of FMAs was not assessed. Another significant limitation is the fact that the EM milestones themselves have not been fully validated [11,12]. …”

Have First-Year Emergency Medicine Residents Achieved Level 1 on Care-Based Milestones?

“…Although the milestones themselves have been assigned specific levels by program directors and core faculty, 6 assessment tools for the milestones have not yet been well studied. Five program directors with more than 30 combined years of experience collaborated to construct the assessment tool and to provide evidence for content validity.…”

Supplemental Milestones for Emergency Medicine Residency Programs: A Validation Study

“…The ACGME, the Residency Review Committee for Emergency Medicine (RRC-EM), and the American Board of Emergency Medicine (ABEM) co-convened the Emergency Medicine (EM) Milestones Working Group to create the EM Milestones 3,4. As described by ABEM, “the EM Milestones are a matrix of the knowledge, skills, abilities, attitudes, and experiences that should be acquired during specialty training in EM.”5 Validated and published in 2013, the EM Milestones are used to track and report residents’ progress in 23 different content domains described as subcompetencies 3,4,6…”