Agreement of Program Directors With Clinical Competency Committees for Fellow Entrustment

Abstract

Objectives: Fellowship program directors (FPD) and Clinical Competency Committees (CCCs) both assess fellow performance. We examined the association of entrustment levels determined by the FPD with those of the CCC for 6 common pediatric subspecialty entrustable professional activities (EPAs), hypothesizing there would be strong correlation and minimal bias between these raters. Methods: The FPDs and CCCs separately assigned a level of supervision to each of their fellows for 6 common pediatric subspecialty EPAs. For each EPA, we determined the correlation between FPD and CCC assessments and calculated bias as CCC minus FPD values for when the FPD was or was not a member of the CCC. In addition, we examined the effect of program size, FPD understanding of EPAs, and subspecialty on the correlations. Data were obtained in fall 2014 and spring 2015. Results: A total of 1040 fellows were assessed in the fall and 1048 in the spring. In both periods and for each EPA, there was a strong correlation between FPD and CCC supervision levels ( P < .001). The correlation was somewhat lower when the FPD was not a CCC member ( P < .001). Overall bias in both periods was small. Conclusions: The correlation between FPD and CCC assignment of EPA supervision levels is strong. Although slightly weaker when the FPD is not a CCC member, bias is small, so this is likely unimportant in determining fellow entrustment level. The similar performance ratings of FPDs and CCCs support the validity argument for EPAs as competency-based assessment tools.

Continued Supervision for the Common Pediatric Subspecialty Entrustable Professional Activities May Be Needed Following Fellowship Graduation

“…[7][8][9][10][11] An exemplar specialty and subspecialty initiative is pediatrics, where there is a growing body of literature demonstrating the effectiveness of EPAs as a program of assessment for graduate medical education (GME). [12][13][14][15] In the United States pediatric subspecialty training, 3 to 6 EPAs have been developed for each individual subspecialty (74 total), and there are an additional 7 EPAs covering content relevant for all pediatric subspecialties (see Table 1). 8,13 Using the pediatric subspecialty EPAs to assess fellows provides an opportunity to ensure fellowship graduates achieve an expected level of supervision (LOS); however, these expected levels are not currently agreed upon.…”

“…An exemplar specialty and subspecialty initiative is pediatrics, where there is a growing body of literature demonstrating the effectiveness of EPAs as a program of assessment for graduate medical education (GME). 12–15 In the United States pediatric subspecialty training, 3 to 6 EPAs have been developed for each individual subspecialty (74 total), and there are an additional 7 EPAs covering content relevant for all pediatric subspecialties (see Table 1). 8,13…”

Establishing Entrustable Professional Activities in Pediatric Cardiac Critical Care*

“…In a subspecialty with many stakeholders such as PCCC, there are limitations to autonomy in complex decisions for trainees, and some EPAs are infrequently performed in observed settings during fellowship training. Recognizing these limitations, programs often use grouped assessments of faculty observations to inform the level of entrustment and development of the trainee over time (14). Additionally, programs may assign progressively more responsibility to more advanced PCCC trainees within a supervised structure to allow for some observation of these EPAs.…”

A realist synthesis of prospective entrustment decision making by entrustment or clinical competency committees

“…63 Time, energy, space, workload, trainees to review and engagement are important factors during E/CCC work. 36,47,51,62,64,68 During review, attention should be paid to managing biases, 58,61,69 ensuring that trainees with disabilities are treated equitably while also ensuring privacy of personal health issues, 66 and avoiding untoward effects of hierarchy on the CCC. 58,64 Finally, data visualisation can enable and offset cognitive load of E/CCC members when making entrustment decisions.…”