Entrustment Decision Making in the Core Entrustable Professional Activities: Results of a Multi-Institutional Study

Abstract

Purpose In 2014, the Association of American Medical Colleges defined 13 Core Entrustable Professional Activities (EPAs) that all graduating students should be ready to do with indirect supervision upon entering residency and commissioned a 10-school, 5-year pilot to test implementing the Core EPAs framework. In 2019, pilot schools convened trained entrustment groups (TEGs) to review assessment data and render theoretical summative entrustment decisions for class of 2019 graduates. Results were examined to determine the extent to which entrustment decisions could be made and the nature of these decisions. Method For each EPA considered (4–13 per student), TEGs recorded an entrustment determination (ready, progressing but not yet ready, evidence against student progressing, could not make a decision); confidence in that determination (none, low, moderate, high); and the number of workplace-based assessments (WBAs) considered (0–>15) per determination. These individual student-level data were de-identified and merged into a multischool database; chi-square analysis tested the significance of associations between variables. Results The 2,415 EPA-specific determinations (for 349 students by 4 participating schools) resulted in a decision of ready (n = 997/2,415; 41.3%), progressing but not yet ready (n = 558/2,415; 23.1%), or evidence against student progression (n = 175/2,415; 7.2%). No decision could be made for the remaining 28.4% (685/2,415), generally for lack of data. Entrustment determinations’ distribution varied across EPAs (chi-square P < .001) and, for 10/13 EPAs, WBA availability was associated with making (vs not making) entrustment decisions (each chi-square P < .05). Conclusions TEGs were able to make many decisions about readiness for indirect supervision; yet less than half of determinations resulted in a decision of readiness to perform this EPA with indirect supervision. More work is needed at the 10 schools to enable authentic summative entrustment in the Core EPAs framework.

Comparing Entrustment Decision-Making Outcomes of the Core Entrustable Professional Activities Pilot, 2019-2020

“…The entrustment workgroup of the pilot used an iterative process of discussions, data collection, and reflection to describe principles, plans, and activities related to the entrustment process; choices schools made about the entrustment process, why choices were made, and challenges with the entrustment process; data considered for each EPA; and results of entrustment decision-making for the first graduating cohort …”

“…Implementing a centralized entrustment process at a medical school involves a major curricular change and poses a variety of cultural, logistical, analytic, psychometric, and ethical challenges . Evaluation of entrustment decision-making for the 2019 graduating cohort highlighted multifactorial challenges in assessment of some of these activities in the workplace, suggesting the need to consider curriculum content revisions and increase availability of WBAs and other assessment data.…”

“…Based on preliminary analysis of volumes of WBAs available, we created a dichotomous variable for WBAs available (0-3 vs ≥4). Schools also incorporated other available assessments into the entrustment process, as described previously …”

“…Lomis et al described a principal aim of the pilot: to develop theoretical summative determinations of each student’s readiness to perform 13 Core EPAs in 2019 “to inform the feasibility of implementing the Core EPAs construct in UME programs.” In an evaluation of decision-making for the 2019 graduating cohort, fewer than half of EPA-specific entrustment determinations had a finding of “ready for indirect supervision,” and more than one-quarter had a report of “could not make an entrustment determination.” This study aimed to evaluate what further progress was made with the 2020 graduating cohort in implementing programmatic assessment and establishing a summative entrustment process using AAMC Core EPAs.…”

Core Entrustable Professional Activities for Entering Residency: A National Survey of Graduating Medical Students’ Self-Assessed Skills by Specialty

“… 2 , 42 Despite limitations of self-assessment data, we do note that studies have shown similarities in patterns across various EPAs—to patterns we observed across EPAs in self-assessed skills in both high SAA and moderate SAA groups—in assessments by others of students’ readiness to perform EPAs under indirect supervision. 32 , 42 , 43 …”