Clinical Competency Committees and Assessment: Changing the Conversation in Graduate Medical Education

Colbert, Colleen Y.; Dannefer, Elaine F.; French, Judith C.

doi:10.4300/jgme-d-14-00448.1

Cited by 38 publications

(29 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The results of our study support the findings of other studies that suggest that the socially situated professional judgements of CCC members are vital to both the integrity of the evidentiary review process and the determination of decisions on a trainee's progression . In light of this, the effortless data interpretation processes we have described may not be as uncomplicated as they appear to be on the surface, and may operate to obscure limitations of the assessment data on which the decision is based.…”

Section: Discussionsupporting

confidence: 85%

“…In addition to prompting us to reconsider what ‘counts’ as evidence, the results of our study have some practical implications. Given their intimate knowledge of assessment data, it has been speculated that CCCs are uniquely positioned to identify gaps or limitations in programmes of assessment . The results of this study highlight how this occurs in practice.…”

Section: Discussionmentioning

confidence: 84%

“…This process relies on the expertise and judgement of committee members and their ability to interpret performance evidence and transform it into a decision about trainee competence. The process of synthesising heterogeneous assessment data is fraught with challenges and the need for efficient, creative strategies to support the complex task of data aggregation has been communicated in the literature …”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Some assembly required: tracing the interpretative work of Clinical Competency Committees

et al. 2019

View full text Add to dashboard Cite

Objectives This qualitative study describes the social processes of evidence interpretation employed by Clinical Competency Committees (CCCs), explicating how they interpret, grapple with and weigh assessment data. Methods Over 8 months, two researchers observed 10 CCC meetings across four postgraduate programmes at a Canadian medical school, spanning over 25 hours and 100 individual decisions. After each CCC meeting, a semi‐structured interview was conducted with one member. Following constructivist grounded theory methodology, data collection and inductive analysis were conducted iteratively. Results Members of the CCCs held an assumption that they would be presented with high‐quality assessment data that would enable them to make systematic and transparent decisions. This assumption was frequently challenged by the discovery of what we have termed ‘problematic evidence’ (evidence that CCC members struggled to meaningful interpret) within the catalogue of learner data. When CCCs were confronted with ‘problematic evidence’, they engaged in lengthy, effortful discussions aided by contextual data in order to make meaning of the evidence in question. This process of effortful discussion enabled CCCs to arrive at progression decisions that were informed by, rather than ignored, problematic evidence. Conclusions Small groups involved in the review of trainee assessment data should be prepared to encounter evidence that is uncertain, absent, incomplete, or otherwise difficult to interpret, and should openly discuss strategies for addressing these challenges. The answer to the problem of effortful processes of data interpretation and problematic evidence is not as simple as generating more data with strong psychometric properties. Rather, it involves grappling with the discrepancies between our interpretive frameworks and the inescapably subjective nature of assessment data and judgement.

show abstract

Section: Discussionsupporting

confidence: 85%

Section: Discussionmentioning

confidence: 84%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Some assembly required: tracing the interpretative work of Clinical Competency Committees

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Importantly, our study contributes to the literature on CCs by providing empirical evidence that supports the claim that CCs’ knowledge of assessment data can be harnessed to identify points at which the programme of assessment is not unfolding as anticipated 10,48,49 and to generate tailored solutions. Although our work provides insight into the types of quality improvement work in which CCs are engaged, we still have much to learn about the effects of this work.…”

Section: Discussionsupporting

confidence: 56%

Beyond summative decision making: Illuminating the broader roles of competence committees

et al. 2020

View full text Add to dashboard Cite

Context: Although competence committees (CCs) are most often conceptualised as group decision-making bodies, policy documents forward a more ambitious vision and outline several additional roles for CCs that support the continuous quality improvement of education programmes and promote formative feedback. However, whether these functions are encompassed in the everyday work of CCs is currently unknown.Methods: This constructivist grounded theory study elaborates the range of roles taken up by CCs and illuminates the processes through which these roles are actualised. Two investigators observed 27 CC meetings (>80 hours) across seven diverse postgraduate programmes at a single Canadian institution. Following each observation, a semi-structured interview was conducted with one CC member. Data collection and analysis unfolded iteratively. Results:In this study, CCs did much more than make summative decisions about progression and entrustment; they identified a variety of problems that arose at the levels of both the individual learner and the programme, and leveraged their knowledge of assessment data to develop solutions. The problem-solving work of CCs was enabled by the in-depth data review, synthesis and analysis work that occurred before scheduled meetings, outside protected academic or administrative time. Although this work often appeared invisible to those outside the committee, the insights gleaned from data review provided committee members with essential information about how their programme of assessment was unfolding in practice. Conclusions:Competence committees may be an untapped resource that can support assessment for learning, local evaluation of competency-based medical education (CBME) implementation and continuous quality improvement for programmes of assessment. However, the ability of CCs to engage in this work is fragile and is currently sustained by the willingness of faculty members to devote their time and energy to it. The resourcing of CCs may have profound implications for translation of the theory of programmatic assessment and CBME into practice. 518 | PACK et Al. | INTRODUC TI ONCompetence committees (CCs) are a recent addition to postgraduate medical education. In the literature, these small groups are most often defined as summative decision-making bodies, responsible for the review, collation and evaluation of trainee assessment data. 1-4 To date, this body of literature has been organised around two themes: (a) developing best practices and forwarding practical advice for the assemblage and operation of CCs, 5-9 and (b) understanding how the decision-making work of the CC unfolds. 1,[10][11][12][13][14] The latter literature has a strong emphasis on summative decision making, explicating how these small groups collate and interpret aggregate assessment data to inform decisions about progression and entrustment. 1,13,15,16 Given the high-stakes nature of these summative decisions, it is unsurprising that they have been subject to significant scholarly interest. However, the focus ...

show abstract

“…Clinical competency committees are small groups entrusted with these analytic and decision-making tasks [5, 6]. These committees play a dual role, as both gatekeepers and learning supports, ensuring patient safety and guiding trainees [7–9].…”

Section: Introductionmentioning

confidence: 99%

How do small groups make decisions?: A theoretical framework to inform the implementation and study of clinical competency committees

Chahine¹,

Cristancho²,

Padgett³

et al. 2017

Perspect Med Educ

View full text Add to dashboard Cite

In the competency-based medical education (CBME) approach, clinical competency committees are responsible for making decisions about trainees’ competence. However, we currently lack a theoretical model for group decision-making to inform this emerging assessment phenomenon. This paper proposes an organizing framework to study and guide the decision-making processes of clinical competency committees.This is an explanatory, non-exhaustive review, tailored to identify relevant theoretical and evidence-based papers related to small group decision-making. The search was conducted using Google Scholar, Web of Science, MEDLINE, ERIC, and PsycINFO for relevant literature. Using a thematic analysis, two researchers (SC & JP) met four times between April–June 2016 to consolidate the literature included in this review.Three theoretical orientations towards group decision-making emerged from the review: schema, constructivist, and social influence. Schema orientations focus on how groups use algorithms for decision-making. Constructivist orientations focus on how groups construct their shared understanding. Social influence orientations focus on how individual members influence the group’s perspective on a decision. Moderators of decision-making relevant to all orientations include: guidelines, stressors, authority, and leadership.Clinical competency committees are the mechanisms by which groups of clinicians will be in charge of interpreting multiple assessment data points and coming to a shared decision about trainee competence. The way in which these committees make decisions can have huge implications for trainee progression and, ultimately, patient care. Therefore, there is a pressing need to build the science of how such group decision-making works in practice. This synthesis suggests a preliminary organizing framework that can be used in the implementation and study of clinical competency committees.

show abstract

Clinical Competency Committees and Assessment: Changing the Conversation in Graduate Medical Education

Cited by 38 publications

References 20 publications

Some assembly required: tracing the interpretative work of Clinical Competency Committees

Some assembly required: tracing the interpretative work of Clinical Competency Committees

Beyond summative decision making: Illuminating the broader roles of competence committees

How do small groups make decisions?: A theoretical framework to inform the implementation and study of clinical competency committees

Contact Info

Product

Resources

About