Validity Evidence for Assessing Entrustable Professional Activities During Undergraduate Medical Education

Abstract

Purpose To explore validity evidence for the use of entrustable professional activities (EPAs) as an assessment framework in medical education. Method Formative assessments on the 13 Core EPAs for entering residency were collected for 4 cohorts of students over a 9- to 12-month longitudinal integrated clerkship as part of the Education in Pediatrics Across the Continuum pilot at the University of Minnesota Medical School. The students requested assessments from clinical supervisors based on direct observation while engaging in patient care together. Based on each observation, the faculty member rated the student on a 9-point scale corresponding to levels of supervision required. Six EPAs were included in the present analyses. Student ratings were depicted as curves describing their performance over time; regression models were employed to fit the curves. The unit of analyses for the learning curves was observations rather than individual students. Results (1) Frequent assessments on EPAs provided a developmental picture of competence consistent with the negative exponential learning curve theory; (2) This finding was true across a variety of EPAs and across students; and (3) The time to attain the threshold level of performance on the EPA for entrustment varied by student and EPA. Conclusions The results provide validity evidence for an EPA-based program of assessment. Students assessed using multiple observations performing the Core EPAs for entering residency demonstrate classic developmental progression toward the desired level of competence resulting in entrustment decisions. Future work with larger data samples will allow further psychometric analyses of assessment of EPAs.

Development of entrustable professional activities for regional anesthesia and pain medicine fellowship training

“…Further, supervision requirements significantly decreased with increased level and duration of training 5. In addition, a follow-up study of the undergraduate medical education EPA pilot program resulted in substantive evidence that EPAs provide valid information on progression of trainees towards level of competence 16. Others have also found supporting evidence to validate EPAs as a method of assessment 17 18.…”

Longitudinal assessment of competency development at The Ohio State University using the competency-based veterinary education (CBVE) model

“…A recent paper described that validity evidence for an assessment system can be supported with demonstration that growth in performance over time follows a theoretically predictable pattern (such as learning curves) ( 9 ). Figure 3 shows such patterns for all domains of competence that were assessed in learners at OSU-CVM using the CBVE.…”

“…What is presented in this study are the preliminary building blocks for a programmatic approach ( 8 ) where assessment data are collected longitudinally and periodically reviewed by an oversight clinical educators' committee that renders high-stakes progress decisions regarding learners ( 5 ). Here, we report on the use of two assessment methods that incorporate scales which have been reported to result in more reliable and valid scores over time ( 9 ). The intent of using these data are to provide students and stakeholders with feedback regarding competency progression through clinical rotations.…”

Using entrustable professional activities to better prepare students for their postgraduatemedical training: A medical student’s perspective

“…One plausible explanation for this is that the EPA framework gives evaluators a shared mental model of standardized student skills and thus facilitates assessments of students even if faculty only work with a student during a single clinic day. The results from this same medical school’s EPA assessment program also demonstrated external validity [ 41 ], demonstrating the potential for EPAs to improve both feedback quantity and reliability.…”