“…8 Program directors were queried about how frequently residents’ preoperative assessments and operative plans were modified by the attending surgeon using the following scale: 1-always, 2-frequently, 3-occasionally, 4-rarely, 5-never. The Resident Entrustability Index 8 was used with a score range of 1-5; a score of 4 or 5 was considered “entrustable” since this encompassed a resident who was trusted to execute that component of an EPA without supervision or to supervise others in the practice of that component (see Figure 1 for a depiction of the Resident Entrustability Index).Figure 1.Resident Entrustment Index. A score of 4 or 5 was considered “entrustable” while a score of 1-3 was considered “not entrustable.”…”
Introduction A 2020 survey of post-graduate year 5 (PGY5) general surgery residents linked to the American Board of Surgery In-Training Examination (ABSITE) revealed significant deficits in self-efficacy (SE), or personal judgment of one’s ability to complete a task, for 10 commonly performed operations. Identifying whether this deficit is similarly perceived by program directors (PDs) has not been well established. We hypothesized that PDs would perceive higher levels of operative SE compared to PGY5s. Methods A survey was distributed through the Association of Program Directors in Surgery listserv; PDs were queried about their PGY5 residents’ ability to perform the same 10 operations independently and their accuracy of patient assessments and operative plans for components of several core entrustable professional activities (EPAs). Results of this survey were compared to PGY5 residents’ perception of their SE and entrustment based on the 2020 post-ABSITE survey. Chi-squared tests were used for statistical analysis. Results 108 responses were received, representing ∼32% (108/342) of general surgery programs. Perceptions from PDs of PGY5 residents’ operative SE were highly concordant with resident perceptions; no significant differences were observed for 9 of 10 procedures. Both PGY5 residents and PDs perceived adequate levels of entrustment; no significant differences were observed for 6 of 8 EPA components. Conclusions These findings show concordance between PDs and PGY5 residents in their perceptions of operative SE and entrustment. Though both groups perceive adequate levels of entrustment, PDs corroborate the previously described operative SE deficit, illustrating the importance of improved preparation for independent practice.
“…8 Program directors were queried about how frequently residents’ preoperative assessments and operative plans were modified by the attending surgeon using the following scale: 1-always, 2-frequently, 3-occasionally, 4-rarely, 5-never. The Resident Entrustability Index 8 was used with a score range of 1-5; a score of 4 or 5 was considered “entrustable” since this encompassed a resident who was trusted to execute that component of an EPA without supervision or to supervise others in the practice of that component (see Figure 1 for a depiction of the Resident Entrustability Index).Figure 1.Resident Entrustment Index. A score of 4 or 5 was considered “entrustable” while a score of 1-3 was considered “not entrustable.”…”
Introduction A 2020 survey of post-graduate year 5 (PGY5) general surgery residents linked to the American Board of Surgery In-Training Examination (ABSITE) revealed significant deficits in self-efficacy (SE), or personal judgment of one’s ability to complete a task, for 10 commonly performed operations. Identifying whether this deficit is similarly perceived by program directors (PDs) has not been well established. We hypothesized that PDs would perceive higher levels of operative SE compared to PGY5s. Methods A survey was distributed through the Association of Program Directors in Surgery listserv; PDs were queried about their PGY5 residents’ ability to perform the same 10 operations independently and their accuracy of patient assessments and operative plans for components of several core entrustable professional activities (EPAs). Results of this survey were compared to PGY5 residents’ perception of their SE and entrustment based on the 2020 post-ABSITE survey. Chi-squared tests were used for statistical analysis. Results 108 responses were received, representing ∼32% (108/342) of general surgery programs. Perceptions from PDs of PGY5 residents’ operative SE were highly concordant with resident perceptions; no significant differences were observed for 9 of 10 procedures. Both PGY5 residents and PDs perceived adequate levels of entrustment; no significant differences were observed for 6 of 8 EPA components. Conclusions These findings show concordance between PDs and PGY5 residents in their perceptions of operative SE and entrustment. Though both groups perceive adequate levels of entrustment, PDs corroborate the previously described operative SE deficit, illustrating the importance of improved preparation for independent practice.
“…Another solution is the universal standardization of clerkship grading systems, terms, and evaluations. Some have called for a complete switch to competency-based assessments, as is occurring in residencies, [51][52][53] that focus on mastery of material and areas important to PDs, such as leadership Scholarly Perspective Academic Medicine, Vol. 98, No.…”
Section: Potential Solutionsmentioning
confidence: 99%
“…Another solution is the universal standardization of clerkship grading systems, terms, and evaluations. Some have called for a complete switch to competency-based assessments, as is occurring in residencies, 51–53 that focus on mastery of material and areas important to PDs, such as leadership and interpersonal communication, 54–57 rather than grades. These approaches have recently been explored in the context of Core Entrustable Professional Activities for Entering Residency (CEPAR).…”
Clerkship grading is a core feature of evaluation for medical students’ skills as physicians and is considered by most residency program directors to be an indicator of future performance and success. With the transition of the U.S. Medical Licensing Examination Step 1 score to pass/fail, there will likely be even greater reliance on clerkship grades, which raises several important issues that need to be urgently addressed. This article details the current landscape of clerkship grading and the systemic discrepancies in assessment and allocation of honors. The authors examine not only objectivity and fairness in clerkship grading but also the reliability of clerkship grading in predicting residency performance and the potential benefits and drawbacks to adoption of a pass/fail clinical clerkship grading system. In the promotion of a more fair and equitable residency selection process, there must be standardization of grading systems with consideration of explicit grading criteria, grading committees, and/or structured education of evaluators and assessors regarding implicit bias. In addition, greater adherence and enforcement of transparency in grade distributions in the Medical Student Performance Evaluation is needed. These changes have the potential to level the playing field, foster equitable comparisons, and ultimately add more fairness to the residency selection process.
“…Determining best practices for trainee performance evaluation using EPAs, and the practical utility of these evaluations, is an area of active investigation. [2][3][4] The study by Krumm et al 1 provides evidence of the reliability of a WBA-based assessment tool in predicting future performance. While it may seem obvious that past performance predicts future success, EPAs inherently rely on this assumption and that demonstrated competency will persist throughout a surgical career.…”
Section: Invited Commentarymentioning
confidence: 99%
“…One challenge with EPAs is that they rely on an assessment by one faculty surgeon that must then be translatable and reliable to others. Determining best practices for trainee performance evaluation using EPAs, and the practical utility of these evaluations, is an area of active investigation . The study by Krumm et al provides evidence of the reliability of a WBA-based assessment tool in predicting future performance.…”
Surgical education is rife with assessment tools and feedback mechanisms. Are these evaluations and instruments useful? Do they predict ongoing competency and practice readiness?With a transition to entrustable professional activities (EPAs) looming, these are vital questions to answer. Krumm and colleagues 1 provide important evidence for the use of workplace-based assessments (WBAs) as a tool for predicting the future performance of surgical trainees. They used longitudinal WBAs, c aptured by the SIMPL application (PLS Collaborative), to quantify a surgical trainee's current competence with a specific procedure and create a model that predicts future performance. Using a robust data set of 63 248 SIMPL evaluations across 70 general surgery programs and 193 procedures, the authors developed a predictive model based on specific procedure, postgraduate year level, month of the academic year, and prior procedurespecific trainee successes. This study provides a significant contribution to the field of surgical education and performance assessment by translating WBAs into a metric that can reliably predict future trainee performance without relying on reported case numbers as a surrogate for competence.There is ongoing discussion regarding the best methods to evaluate surgical trainees for competence, with movement toward the use of EPAs. 2 One challenge with EPAs is that they rely on an assessment by one faculty surgeon that must then be translatable and reliable to others. Determining best practices for trainee per-
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
