Reviewing Residents’ Competence

Hauer, Karen E.; Chesluk, Benjamin; Iobst, William; Holmboe, Eric S.; Baron, Robert B.; Boscardin, Christy; Cate, Olle ten; O’Sullivan, Patricia

doi:10.1097/acm.0000000000000736

Cited by 98 publications

(57 citation statements)

References 19 publications

(17 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The large volume of diverse data produced, in turn, requires new processes of assembly and analysis. In response to this need, postgraduate medical programmes in Canada and the USA have integrated new group decision‐making bodies, referred to as Clinical Competency Committees (CCCs), into their programmes of assessment . These small groups operate as centralised review bodies that conduct in‐depth reviews of diverse assessment data collected about trainee performance and use collective processes to render decisions about progression …”

Section: Introductionmentioning

confidence: 99%

“…This type of small group decision making is no small task. Committees must sift through large amounts of quantitative and qualitative performance data, interpret their meanings, and create an intelligible composite that reliably communicates the trainee's competencies . This process relies on the expertise and judgement of committee members and their ability to interpret performance evidence and transform it into a decision about trainee competence.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Some assembly required: tracing the interpretative work of Clinical Competency Committees

et al. 2019

View full text Add to dashboard Cite

Objectives This qualitative study describes the social processes of evidence interpretation employed by Clinical Competency Committees (CCCs), explicating how they interpret, grapple with and weigh assessment data. Methods Over 8 months, two researchers observed 10 CCC meetings across four postgraduate programmes at a Canadian medical school, spanning over 25 hours and 100 individual decisions. After each CCC meeting, a semi‐structured interview was conducted with one member. Following constructivist grounded theory methodology, data collection and inductive analysis were conducted iteratively. Results Members of the CCCs held an assumption that they would be presented with high‐quality assessment data that would enable them to make systematic and transparent decisions. This assumption was frequently challenged by the discovery of what we have termed ‘problematic evidence’ (evidence that CCC members struggled to meaningful interpret) within the catalogue of learner data. When CCCs were confronted with ‘problematic evidence’, they engaged in lengthy, effortful discussions aided by contextual data in order to make meaning of the evidence in question. This process of effortful discussion enabled CCCs to arrive at progression decisions that were informed by, rather than ignored, problematic evidence. Conclusions Small groups involved in the review of trainee assessment data should be prepared to encounter evidence that is uncertain, absent, incomplete, or otherwise difficult to interpret, and should openly discuss strategies for addressing these challenges. The answer to the problem of effortful processes of data interpretation and problematic evidence is not as simple as generating more data with strong psychometric properties. Rather, it involves grappling with the discrepancies between our interpretive frameworks and the inescapably subjective nature of assessment data and judgement.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Some assembly required: tracing the interpretative work of Clinical Competency Committees

et al. 2019

View full text Add to dashboard Cite

show abstract

“…A 2015 qualitative analysis of 34 CCCs by Hauer et al (8) found that most committees were oriented toward a ‘problem-identification’ model, in which the committee members’ efforts focused on searching performance data for ‘red-flags’ to identify struggling residents. In this model, records of residents with performance issues were reviewed in detail for other events, while residents without issues were minimally reviewed, if at all.…”

Section: The Problem and Ways To Approach Itmentioning

confidence: 99%

“…However, a few programs pursued what was characterized as a ‘developmental’ model, in which CCCs compared all residents’ performance data against established benchmarks, with focused discussion on areas for resident growth. Those using a developmental model identified concerns of increased time for the CCCs to apply the complex milestone format and the need for faculty development in qualitative methods as potential implementation challenges (8). …”

Section: The Problem and Ways To Approach Itmentioning

confidence: 99%

“…However, any assessment standardization may inadvertently conceal important individual differences between trainees, making a program less flexible for its higher and lower performing learners (3). Furthermore, in more complex summative evaluation systems, formative feedback may be misperceived as summative which could threaten assessor–learner relationships, possibly resulting in range restriction or failure of faculty to record written constructive feedback (8). …”

Section: Potential Pitfalls In CCC Creationmentioning

confidence: 99%

See 1 more Smart Citation

Design of a clinical competency committee to maximize formative feedback

Donato

Alweis

Wenderoth

2016

Journal of Community Hospital Internal Medicine Perspectives

View full text Add to dashboard Cite

BackgroundAs the next phase in the roll-out of Next Accreditation System, US residency programs are to develop Clinical Competency Committees (CCCs) to formally implement outcome-based medical education objectives in the resident assessment process. However, any changes to an assessment system must consider balancing formative and summative tensions, flexibility and standardization tensions, fairness and transparency to learners, and administrative burden for faculty.Objectives/MethodsIn this article, one program discusses the approach one internal medicine residency took to create a developmental model CCC. In this model, a learner's mentor presents the argument for competence to the CCC, while a second reviewer presents challenges to that argument to the rest of the committee members. The CCC members provide other insights and make recommendations. The mentor presents the final committee recommendations to that resident, who then works with the mentor to develop a plan for future action.ResultsCCC second reviewers spent an average of 30.4 min (SD: 11.4) preparing for each resident's discussion, a duty performed 5–7 times every 6 months. Faculty development was associated with an increase in the number of action-oriented comments in the meeting minutes (3.2–4.1 comments per resident, p=0.001). CCC members and mentors gave higher Likert-type ratings than residents for fairness (4.8 vs. 4.0) and learning prioritization (4.7 vs. 4.2), but similar ratings for transparency (4.0 vs. 4.2).ConclusionDevelopmental model CCCs may be feasible for residency programs, but faculty development may be necessary.

show abstract