The Current State of Core Competency Assessment in Emergency Medicine and a Future Research Agenda: Recommendations of the Working Group on Assessment of Observable Learner Performance

Kessler, Chad; Leone, Katrina A.

doi:10.1111/acem.12023

Cited by 21 publications

(32 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…7,8,10 Our findings confirm the study by Santen and colleagues 3 showing that incoming EM interns were not taught or assessed on many of the Level 1 milestones. They noted that in the future, medical schools will need to be responsible for cross-cutting milestones that span across specialties, with individual clerkships needing to take responsibility for educating senior students in specialty-specific milestones.…”

Section: Discussionsupporting

confidence: 85%

“…[6][7][8] Direct observation, global assessment, and objective structured clinical examinations (OSCE) are common tools used for resident evaluation. 5,9 While medical educators struggle to determine the utility of individual assessment tools, 7,8,10 van der Vleuten and Schuwirth's 11 work suggests that it is most important to develop a program of assessment with multiple low-stakes assessments. The tools used in multiple low-stakes assessments provide educators with a well-rounded view of learners' performance.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Programmatic Assessment of Level 1 Milestones in Incoming Interns

Hauff

Hopson

Losman

et al. 2014

Academic Emergency Medicine

View full text Add to dashboard Cite

Objectives: With the Accreditation Council for Graduate Medical Education (ACGME) Next Accreditation System, emergency medicine (EM) residency programs will be required to report residents' progress through the EM milestones. The milestones include five progressively advancing skill levels, with Level 1 defining the skill set of a medical school graduate and Level 5, that of an attending physician. The ACGME stresses that multiple forms of assessment should be used to ensure capture of the multifaceted competencies. The objective of this study was to determine the feasibility and results of programmatic assessment of Level 1 milestones using multisource assessments for incoming EM interns in July. Methods:The study population was interns starting in 2012 and 2013. Interns' Level 1 milestone assessment was done with four distinct methods: 1) the postgraduate orientation assessment (POA) by the Graduate Medical Education Office for all incoming interns (this multistation examination covers nine of the EM milestones and includes standardized patient cases, task completion, and computer-based stations); 2) direct observation of patient encounters by core faculty using a milestones-based clinical skills competency checklist; 3) the global monthly assessment at the end of the intern orientation month that was updated to reflect the EM milestones; and 4) faculty assessment during procedural labs. These occurred during the July orientation month that included the POA, clinical shifts, didactic sessions, and procedure labs.Results: In the POA, interns were competent in 48% to 93% of the milestones assessed. Overall, competency was 70% to 80%, with low scores noted in aseptic technique (patient care Milestone 13 [PC13]) and written and verbal hand-off (interpersonal communications skills [ICS]2). In overall communication, 70% of interns demonstrated competency. In excess of 80% demonstrated competency in critical values interpretation (PC3), informed consent (PC9), pain assessment (PC11), and geriatric functional assessment (PC3). On direct observation, almost all Level 1 milestones were achieved (93% to 100%); however, only 78% of interns achieved competency in pharmacotherapy (PC5). On global monthly evaluations, all interns met Level 1 milestones.Conclusions: A multisource assessment of EM milestones is feasible and useful to determine Level 1 milestones achievement for incoming interns. A structured assessment program, used in conjunction with more traditional forms of evaluation such as global monthly evaluations and direct observation, is useful for identifying deficits in new trainees and may be able inform the creation of early intervention programs.ACADEMIC EMERGENCY MEDICINE 2014;21:694-698

show abstract

Section: Discussionsupporting

confidence: 85%

mentioning

confidence: 99%

Programmatic Assessment of Level 1 Milestones in Incoming Interns

Hauff

Hopson

Losman

et al. 2014

Academic Emergency Medicine

View full text Add to dashboard Cite

show abstract

“…5 It also outlines different methods of bedside US competency assessments. How the emergency US milestones affect US rotations remains unknown, and the best way to standardize US competency assessment remains unanswered.…”

Section: Discussionmentioning

confidence: 99%

“…4 This patient care skill requires EM residents to demonstrate competency in performing bedside US. [5][6][7] To successfully implement US milestones, it is crucial to understand the current competency assessment methods used among EM programs including their content, frequency, and specific assessment tools. The objective of this study was to assess the current methods used by EM residency training programs to evaluate resident competency in bedside US.…”

mentioning

confidence: 99%

Ultrasound Competency Assessment in Emergency Medicine Residency Programs

Amini

Adhikari

Fiorello

2014

Academic Emergency Medicine

View full text Add to dashboard Cite

Objectives: In the Model of the Clinical Practice of Emergency Medicine (EM), bedside ultrasound (US) is listed as one of the essential procedural skills. EM milestones released by Accreditation Council for Graduate Medical Education and American Board of Emergency Medicine require residents to demonstrate competency in bedside US. The purpose of this study was to assess the current methods used by EM residency training programs to evaluate resident competency in bedside US.Methods: This was a cross-sectional survey study. A questionnaire on US education and competency assessment was electronically sent to all EM residency program directors and emergency US directors. The survey consisted of questions regarding the US rotation, structure of US curriculum, presence of US fellowship, image archiving, quality assurance methods, feedback, competency assessment tools, and frequency of assessment. The survey responses are reported as the percentages of total respondents along with 95% confidence intervals (CIs).Results: A total of 124 of 161 EM residency programs participated in this study, representing a 77% response rate. Twenty-six percent (95% CI = 18% to 34%) of programs assess competency only at the end of the US rotation. Eight percent (95% CI = 3% to 13%) assess competency only every 6 months, and 13% (95% CI = 7% to 19%) assess competency only annually. Eight percent (95% CI = 3% to 13%) assess competency only during the final year of training. Thirty percent (95% CI = 22% to 38%) of programs assess competency with a combination of the above intervals, and 16% (95% CI = 10% to 22%) do not assess US competency. Fourteen percent (95% CI = 8% to 20%) use objective structured clinical examinations (OSCEs), and 21% (95% CI = 14% to 28%) use standardized direct observation tools (SDOTs) to assess resident competency in US. Approximately one-third (33%, 95% CI = 24% to 41%) of standardized testing for US competency is conducted with multiple-choice questions. Thirty percent (95% CI = 21% to 38%) administer practical examinations to assess US skills.Conclusions: Currently, a majority of EM residency programs assess resident competency in bedside US. However, there is significant variation in the methods of competency assessment. The Accreditation Council for Graduate Medical Education (ACGME) recently introduced the concept of educational milestones for resident assessment, and subsequently the EM Milestones Working Group developed specialty-specific milestones. 4 In October 2012, the ACGME and the American Board of Emergency Medicine (ABEM) finalized 23 milestones, and US was designated patient care skill number 12.4 This patient care skill requires EM residents to demonstrate competency in performing bedside US. [5][6][7] To successfully implement

show abstract

“…21 How many observation assessments are necessary? What types of patient encounters should be observed?…”

mentioning

confidence: 99%