A multispecialty ethnographic study of clinical competency committees (CCCs)

Ekpenyong, Andem; Edgar, Laura; Wilkerson, LuAnn; Holmboe, Eric S.

doi:10.1080/0142159x.2022.2072281

Cited by 5 publications

(5 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, our milestone process fosters a two-way conversation about performance rooted in self-reflection. 26 Self-assessments using milestones can help direct improvement in the many domains and skills required of a competent family physician.…”

Section: Discussionmentioning

confidence: 99%

Resident Self-assessment and Clinical Competency Committee Evaluations in Family Medicine: A Single-Institution Study

Bhardwaj

Porterfield

Kuo

et al. 2023

Fam Med

View full text Add to dashboard Cite

Background and Objectives: We sought to describe the process of integrating resident self-assessments into milestone assessments at the University of Texas Medical Branch Family Medicine Residency Program in Galveston, Texas. We compared resident self-assessments across milestones to Clinical Competency Committee (CCC) assessments across terms (fall versus spring) and by postgraduate year (PGY). Methods: In fall 2020, the milestone assessment process was updated to include a resident milestone self-assessment, which was used as the starting point for CCC assessment. We calculated mean and standard deviation of average milestone scores for both self-assessment and CCC for each PGY. We used repeated measure analysis of variance to examine within- and between-subject effects. Results: Self-assessment and CCC assessments were completed for 30 postgraduate trainees for spring 2020 and fall 2021 terms, for a total of 60 self- and 60 CCC assessments. CCC score was similar to self-assessment. There were larger variations in the resident self-assessment scores than CCC scores. Self-assessment scores increased by PGY, but were not different between fall and spring terms. We found a significant three-way interaction of assessors, terms, and PGYs. Conclusions: Resident milestone self-assessment enables residents to participate in the assessment process, and when differences exist between self- and CCC assessments, specific feedback can be given based on individual milestone skills. Our study showed progression between PGY regardless of the assessor, but only CCC assessment showed significant differences between terms.

show abstract

Section: Discussionmentioning

confidence: 99%

Resident Self-assessment and Clinical Competency Committee Evaluations in Family Medicine: A Single-Institution Study

Bhardwaj

Porterfield

Kuo

et al. 2023

Fam Med

View full text Add to dashboard Cite

show abstract

“…A prior study analyzing milestone response process validity across multiple specialties has also shown wide variability in both CCC composition (e.g., number of members, academic role) and decision-making processes 13 ; this lack of uniformity could contribute to the variation in milestone ratings attributable to factors other than actual trainee performance. Additionally, programs often use different types of assessments—collected at different frequencies and from various types of assessors—to inform milestone ratings, some of which are unique to that program and could create further variability in ratings.…”

Section: Limitations Of Milestones As a Predictive Assessmentmentioning

confidence: 99%

Using Graduating Surgical Resident Milestone Ratings to Predict Patient Outcomes: A Blunt Instrument for a Complex Problem

Montgomery

Lindeman

2023

Academic Medicine

View full text Add to dashboard Cite

In 2013, U.S. general surgery residency programs implemented a milestones assessment framework in an effort to incorporate more competency-focused evaluation methods. Developed by a group of surgical education leaders and other stakeholders working with the Accreditation Council for Graduate Medical Education and recently updated in a version 2.0, the surgery milestones framework is centered around 6 “core competencies”: patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice. While prior work has focused on the validity of milestones as a measure of resident performance, associations between general surgery resident milestone ratings and their post-training patient outcomes have only recently been explored in an analysis in this issue of Academic Medicine by Kendrick et al. Despite their well-designed efforts to tackle this complex problem, no relationships were identified. This accompanying commentary discusses the broader implications for the use of milestone ratings beyond their intended application, alternative assessment methods, and the challenges of developing predictive assessments in the complex setting of surgical care. Although milestone ratings have not been shown to provide the specificity needed to predict clinical outcomes in the complex settings studied by Kendrick et al, hope remains that utilization of other outcomes, assessment frameworks, and data analytic tools could augment these models and further our progress toward a predictive assessment in surgical education. Evaluation of residents in general surgery residency programs has grown both more sophisticated and complicated in the setting of increasing patient and case complexity, constraints on time, and regulation of resident supervision in the operating room. Over the last decade, surgical education research efforts related to resident assessment have focused on measuring performance through accurate and reproducible methods with evidence for their validity, as well as on attempting to refine decision making about resident preparedness for unsupervised practice.

show abstract

“…In a recent multi-institution ethnographic study of CCCs, Ekpenyong and colleagues found substantial variation in CCC processes, with some practices not supported by current evidence for effective group functioning. 52 Importantly, bias can either be mitigated or exacerbated by group process. A growing body of research suggests that high-performing, diverse groups (e.g., groups that implement evidence-based effective team practices) are more likely to make better decisions about learners and better support learners’ professional development.…”

Section: Specific Opportunities To Improve Assessmentmentioning

confidence: 99%

The Urgency of Now: Rethinking and Improving Assessment Practices in Medical Education Programs

et al. 2023

Self Cite

View full text Add to dashboard Cite

Assessment is essential to professional development. Assessment provides the information needed to give feedback, support coaching and the creation of individualized learning plans, inform progress decisions, determine appropriate supervision levels, and, most importantly, help ensure patients and families receive high-quality, safe care in the training environment. While the introduction of competency-based medical education has catalyzed advances in assessment, much work remains to be done. First, becoming a physician (or other health professional) is primarily a developmental process, and assessment programs must be designed using a developmental and growth mindset. Second, medical education programs must have integrated programs of assessment that address the interconnected domains of implicit, explicit and structural bias. Third, improving programs of assessment will require a systems-thinking approach. In this paper, the authors first address these overarching issues as key principles that must be embraced so that training programs may optimize assessment to ensure all learners achieve desired medical education outcomes. The authors then explore specific needs in assessment and provide suggestions to improve assessment practices. This paper is by no means inclusive of all medical education assessment challenges or possible solutions. However, there is a wealth of current assessment research and practice that medical education programs can use to improve educational outcomes and help reduce the harmful effects of bias. The authors’ goal is to help improve and guide innovation in assessment by catalyzing further conversations.

show abstract

A multispecialty ethnographic study of clinical competency committees (CCCs)

Cited by 5 publications

References 24 publications

Resident Self-assessment and Clinical Competency Committee Evaluations in Family Medicine: A Single-Institution Study

Resident Self-assessment and Clinical Competency Committee Evaluations in Family Medicine: A Single-Institution Study

Using Graduating Surgical Resident Milestone Ratings to Predict Patient Outcomes: A Blunt Instrument for a Complex Problem

The Urgency of Now: Rethinking and Improving Assessment Practices in Medical Education Programs

Contact Info

Product

Resources

About