Shortening the Miles to the Milestones: Connecting EPA-Based Evaluations to ACGME Milestone Reports for Internal Medicine Residency Programs

Abstract

The Next Accreditation System requires internal medicine training programs to provide the Accreditation Council for Graduate Medical Education (ACGME) with semiannual information about each resident's progress in 22 subcompetency domains. Evaluation of resident "trustworthiness" in performing entrustable professional activities (EPAs) may offer a more tangible assessment construct than evaluations based on expectations of usual progression toward competence. However, translating results from EPA-based evaluations into ACGME milestone progress reports has proven to be challenging because the constructs that underlay these two systems differ.The authors describe a process to bridge the gap between rotation-specific EPA-based evaluations and ACGME milestone reporting. Developed at the University of Washington in 2012 and 2013, this method involves mapping EPA-based evaluation responses to "milestone elements," the narrative descriptions within the columns of each of the 22 internal medicine subcompetencies. As faculty members complete EPA-based evaluations, the mapped milestone elements are automatically marked as "confirmed." Programs can maintain a database that tallies the number of times each milestone element is confirmed for a resident; these data can be used to produce graphical displays of resident progress along the internal medicine milestones.Using this count of milestone elements allows programs to bridge the gap between faculty assessments of residents based on rotation-specific observed activities and semiannual ACGME reports based on the internal medicine milestones. Although potentially useful for all programs, this method is especially beneficial to large programs where clinical competency committee members may not have the opportunity for direct observation of all residents.

Shadowing emergency medicine residents by medical education specialists to provide feedback on non-medical knowledge-based ACGME sub-competencies

“…Additionally, a study at the University of Mississippi sought to reveal the utility of end-of-shift evaluations of EM residents by clinical faculty. 12 They concluded end-of-shift evaluations “rarely (8%, 372 of 4633) rated a resident as not achieving milestones.” 12 Other studies within anesthesiology, 13 general surgery, 14 and internal medicine 15 residency programs also highlighted the difficulties with evaluating residents and assigning them scores according to the ACGME milestones. Another study by Angus et al 16 surveyed Internal Medicine residents to assess resident perceptions of receiving feedback in the milestone framework.…”

A competency-based approach to critical care education

“…Much of the work in health care entails tasks or responsibilities that must be entrusted to individuals. Research has demonstrated that ability, integrity, and reliability must be fulfilled for someone to trust another person and be willing to be vulnerable to the risks associated with doing so [ 9 ]. An entrusted trainee must demonstrate the necessary competence to execute EPAs unsupervised.…”

“…The difficulty, risk, or complexity of EPAs can be increased based on the ability of trainees. Therefore, medical educators can apply a series of EPAs and supervise the competency level that trainees need to achieve [ 9 ]. EPAs can be executed, and the process and results can be observed separately or simultaneously depending on the competency level of trainees.…”

Group Trust in the Setting of Competency Committee Decisions: A Qualitative Observational Study

“…Built upon the fundamentals of competency-based medical education, the Accreditation Council for Graduate Medical Education (ACGME) Milestones help GME programs determine which of their trainees are on appropriate paths towards unsupervised clinical practice and which are not. Within a GME program, the clinical competency committee (CCC) forms a fundamental part of determining trainees' progress in the context of the Milestones, and some recent studies describe data sources that CCCs use and how they weigh this information in their collective decisions (Choe et al, 2016;Shumway et al, 2015). Several other studies have also looked at group decision making processes and how these concepts may be utilized by CCCs (Chahine et al, 2017;Hauer et al, 2016;Hemmer and Kelly, 2017).…”