Abstract:Background
Competency-based medical education (CBME) was expected to increase the workload of assessment for graduate training programs to support the development of competence. Learning conditions were anticipated to improve through the provision of tailored learning experiences and more frequent, low-stakes assessments. Canada has adopted an approach to CBME called Competence by Design (CBD). However, in the process of implementation, learner anxiety and assessment burden have increased une… Show more
“…These findings are in line with the findings of other work that have identified the unintended burden of assessment within CBME in numerous settings for both faculty and residents. 9 What this work adds to the literature is the identification of both program-specific and cross-program adaptations (proposed and/or enacted) by residency programs related to each of the identified themes. These adaptations were identified by individual programs, factoring in their own contexts and unique characteristics.…”
Section: Discussionmentioning
confidence: 99%
“…These studies have noted a variety of successes and challenges in implementation, but one of the clearest early struggles has been the identification of a major burden of assessment widely experienced by both residents and faculty. 9 Workplace-based assessment is central to CBME and has 2 main purposes. 10 The first is assessment for learning: low-stakes and formative assessment to support learning, the ongoing development of self-assessment skills, and documentation of low-stakes individual data points.…”
Section: Competency-based Medical Educationmentioning
confidence: 99%
“…Furthermore, the concerns related to assessment in CBME are associated, in theory, with concerns about the deterioration of faculty coaching and feedback efforts and their own burnout as well as with issues related to resident wellness. 9 Despite these significant experienced challenges, there is little in the literature describing potential program-level adaptations that could mitigate these risks.…”
mentioning
confidence: 99%
“…These studies have noted a variety of successes and challenges in implementation, but one of the clearest early struggles has been the identification of a major burden of assessment widely experienced by both residents and faculty. 9…”
Residents and faculty have described a burden of assessment related to the implementation of competency-based medical education (CBME), which may undermine its benefits. Although this concerning signal has been identified, little has been done to identify adaptations to address this problem. Grounded in an analysis of an early Canadian pan-institutional CBME adopter’s experience, this article describes postgraduate programs’ adaptations related to the challenges of assessment in CBME. From June 2019–September 2022, 8 residency programs underwent a standardized Rapid Evaluation guided by the Core Components Framework (CCF). Sixty interviews and 18 focus groups were held with invested partners. Transcripts were analyzed abductively using CCF, and ideal implementation was compared with enacted implementation. These findings were then shared back with program leaders, adaptations were subsequently developed, and technical reports were generated for each program. Researchers reviewed the technical reports to identify themes related to the burden of assessment with a subsequent focus on identifying adaptations across programs. Three themes were identified: (1) disparate mental models of assessment processes in CBME, (2) challenges in workplace-based assessment processes, and (3) challenges in performance review and decision making. Theme 1 included entrustment interpretation and lack of shared mindset for performance standards. Adaptations included revising entrustment scales, faculty development, and formalizing resident membership. Theme 2 involved direct observation, timeliness of assessment completion, and feedback quality. Adaptations included alternative assessment strategies beyond entrustable professional activity forms and proactive assessment planning. Theme 3 related to resident data monitoring and competence committee decision making. Adaptations included adding resident representatives to the competence committee and assessment platform enhancements. These adaptations represent responses to the concerning signal of significant burden of assessment within CBME being experienced broadly. The authors hope other programs may learn from their institution’s experience and navigate the CBME-related assessment burden their invested partners may be facing.
“…These findings are in line with the findings of other work that have identified the unintended burden of assessment within CBME in numerous settings for both faculty and residents. 9 What this work adds to the literature is the identification of both program-specific and cross-program adaptations (proposed and/or enacted) by residency programs related to each of the identified themes. These adaptations were identified by individual programs, factoring in their own contexts and unique characteristics.…”
Section: Discussionmentioning
confidence: 99%
“…These studies have noted a variety of successes and challenges in implementation, but one of the clearest early struggles has been the identification of a major burden of assessment widely experienced by both residents and faculty. 9 Workplace-based assessment is central to CBME and has 2 main purposes. 10 The first is assessment for learning: low-stakes and formative assessment to support learning, the ongoing development of self-assessment skills, and documentation of low-stakes individual data points.…”
Section: Competency-based Medical Educationmentioning
confidence: 99%
“…Furthermore, the concerns related to assessment in CBME are associated, in theory, with concerns about the deterioration of faculty coaching and feedback efforts and their own burnout as well as with issues related to resident wellness. 9 Despite these significant experienced challenges, there is little in the literature describing potential program-level adaptations that could mitigate these risks.…”
mentioning
confidence: 99%
“…These studies have noted a variety of successes and challenges in implementation, but one of the clearest early struggles has been the identification of a major burden of assessment widely experienced by both residents and faculty. 9…”
Residents and faculty have described a burden of assessment related to the implementation of competency-based medical education (CBME), which may undermine its benefits. Although this concerning signal has been identified, little has been done to identify adaptations to address this problem. Grounded in an analysis of an early Canadian pan-institutional CBME adopter’s experience, this article describes postgraduate programs’ adaptations related to the challenges of assessment in CBME. From June 2019–September 2022, 8 residency programs underwent a standardized Rapid Evaluation guided by the Core Components Framework (CCF). Sixty interviews and 18 focus groups were held with invested partners. Transcripts were analyzed abductively using CCF, and ideal implementation was compared with enacted implementation. These findings were then shared back with program leaders, adaptations were subsequently developed, and technical reports were generated for each program. Researchers reviewed the technical reports to identify themes related to the burden of assessment with a subsequent focus on identifying adaptations across programs. Three themes were identified: (1) disparate mental models of assessment processes in CBME, (2) challenges in workplace-based assessment processes, and (3) challenges in performance review and decision making. Theme 1 included entrustment interpretation and lack of shared mindset for performance standards. Adaptations included revising entrustment scales, faculty development, and formalizing resident membership. Theme 2 involved direct observation, timeliness of assessment completion, and feedback quality. Adaptations included alternative assessment strategies beyond entrustable professional activity forms and proactive assessment planning. Theme 3 related to resident data monitoring and competence committee decision making. Adaptations included adding resident representatives to the competence committee and assessment platform enhancements. These adaptations represent responses to the concerning signal of significant burden of assessment within CBME being experienced broadly. The authors hope other programs may learn from their institution’s experience and navigate the CBME-related assessment burden their invested partners may be facing.
“…Recently, CBME and EPAs for medical trainee feedback has been adopted in Canada 36. CBME poses a challenge for MSF due to resident and staff physicians feeling overwhelmed by the burden of increased assessment requirements 37. This was reflected in some study participants sharing that mandatory feedback is becoming more of a burden, and some participants want less formalisation of feedback.…”
ObjectivesTo evaluate the impact and feasibility of multisource feedback compared with traditional feedback for trauma team captains (TTCs).DesignA mixed-methods, non-randomised prospective study.SettingA level one trauma centre in Ontario, Canada.ParticipantsPostgraduate medical residents in emergency medicine and general surgery participating as TTCs. Selection was based on a convenience sampling method.InterventionPostgraduate medical residents participating as TTCs received either multisource feedback or standard feedback following trauma cases.Main outcome measuresTTCs completed questionnaires designed to measure the self-reported intention to change practice (catalytic effect), immediately following a trauma case and 3 weeks later. Secondary outcomes included measures of perceived benefit, acceptability, and feasibility from TTCs and other trauma team members.ResultsData were collected following 24 trauma team activations: TTCs from 12 activations received multisource feedback and 12 received standard feedback. The self-reported intention for practice change was not significantly different between groups initially (4.0 vs 4.0, p=0.57) and at 3 weeks (4.0 vs 3.0, p=0.25). Multisource feedback was perceived to be helpful and superior to the existing feedback process. Feasibility was identified as a challenge.ConclusionsThe self-reported intention for practice change was no different for TTCs who received multisource feedback and those who received standard feedback. Multisource feedback was favourably received by trauma team members, and TTCs perceived multisource feedback as useful for their development.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.