Emergency Medicine Residents Consistently Rate Themselves Higher Than Attending Assessments on ACGME Milestones

Abstract

IntroductionIn 2012 the Accreditation Council for Graduate Medical Education (ACGME) introduced the Next Accreditation System (NAS), which implemented milestones to assess the competency of residents and fellows. While attending evaluation and feedback is crucial for resident development, perhaps equally important is a resident’s self-assessment. If a resident does not accurately self-assess, clinical and professional progress may be compromised. The objective of our study was to compare emergency medicine (EM) resident milestone evaluation by EM faculty with the same resident’s self-assessment.MethodsThis is an observational, cross-sectional study that was performed at an academic, four-year EM residency program. Twenty-five randomly chosen residents completed milestone self-assessment using eight ACGME sub-competencies deemed by residency leadership as representative of core EM principles. These residents were also evaluated by 20 faculty members. The milestone levels were evaluated on a nine-point scale. We calculated the average difference between resident self-ratings and faculty ratings, and used sample t-tests to determine statistical significance of the difference in scores.ResultsEighteen residents evaluated themselves. Each resident was assessed by an average of 16 attendings (min=10, max=20). Residents gave themselves statistically significant higher milestone ratings than attendings did for each sub-competency examined (p<0.0001).ConclusionResidents over-estimated their abilities in every sub-competency assessed. This underscores the importance of feedback and assessment transparency. More attention needs to be paid to methods by which residency leadership can make residents’ self-perception of their clinical ability more congruent with that of their teachers and evaluators. The major limitation of our study is small sample size of both residents and attendings.

Critical Appraisal of Emergency Medicine Educational Research: The Best Publications of 2015

“…A total of 434 papers and abstracts satisfied the search criteria, and 61 papers met the inclusion criteria . The authors scored these 61 manuscripts, and the 10 highest scoring quantitative and two qualitative articles are reviewed below, in alphabetical order by first author's last name.…”

Designing and Implementing Emergency Department Pain Management Curriculum: A Delphi Approach

“…Currently, many programs use end‐of‐shift evaluations or self‐evaluations to measure proficiency in these milestones. Unfortunately, these two types of evaluation may not accurately reflect the residents’ proficiency level . Both types of measures tend to overestimate proficiency level.…”

Are All Competencies Equal in the Eyes of Residents? A Multicenter Study of Emergency Medicine Residents’ Interest in Feedback

“…This model is the main assessment framework of the NAS. Physicians are expected to progress through the milestone levels of each competency / sub-competency from novice intern to expert 2,7–10…”