How Should the ACGME Core Competencies Be Measured?

Bell, Hershey S.

doi:10.1097/acm.0b013e3181b17ae6

Cited by 3 publications

(3 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Milestone system, which now delineates 25 subcompetencies spread among the 6 core competencies, individual absolute rankings are more desired than peer-relative or training yearrelative rankings, because the goal is individual progression toward unsupervised practice with recognition that trainees may progress on different learning trajectories. 1 This finding may be of particular importance to training programs across the country, because program directors have been given no specific direction on how to implement evaluation schemas in the Milestone era, and numerous questions remain and are debated. For instance, should the subcompetencies be used verbatim as an assessment tool?…”

Section: Educationmentioning

confidence: 99%

Implementation and Evaluation of the Z-Score System for Normalizing Residency Evaluations

Wanderer

Filho

Rothman³

et al. 2018

Anesthesiology

View full text Add to dashboard Cite

Background Assessment of clinical competence is essential for residency programs and should be guided by valid, reliable measurements. We implemented Baker’s Z-score system, which produces measures of traditional core competency assessments and clinical performance summative scores. Our goal was to validate use of summative scores and estimate the number of evaluations needed for reliable measures. Methods We performed generalizability studies to estimate the variance components of raw and Z-transformed absolute and peer-relative scores and decision studies to estimate the evaluations needed to produce at least 90% reliable measures for classification and for high-stakes decisions. A subset of evaluations was selected representing residents who were evaluated frequently by faculty who provided the majority of evaluations. Variance components were estimated using ANOVA. Results Principal component extraction from 8,754 complete evaluations demonstrated that a single factor explained 91 and 85% of variance for absolute and peer-relative scores, respectively. In total, 1,200 evaluations were selected for generalizability and decision studies. The major variance component for all scores was resident interaction with measurement occasions. Variance due to the resident component was strongest with raw scores, where 30 evaluation occasions produced 90% reliable measurements with absolute scores and 58 for peer-relative scores. For Z-transformed scores, 57 evaluation occasions produced 90% reliable measurements with absolute scores and 55 for peer-relative scores. The results were similar for high-stakes decisions. Conclusions The Baker system produced moderately reliable measures at our institution, suggesting that it may be generalizable to other training programs. Raw absolute scores required few assessment occasions to achieve 90% reliable measurements.

show abstract

Section: Educationmentioning

confidence: 99%

Implementation and Evaluation of the Z-Score System for Normalizing Residency Evaluations

Wanderer

Filho

Rothman³

et al. 2018

Anesthesiology

View full text Add to dashboard Cite

show abstract

“…First, medical curriculum must have clear learning objectives and criteria. 24 Key players such as trainees and faculty must be previously aware of the objectives. Once taught, the trainees' evaluation must be based on what was taught and must be clearly documented.…”

Section: General Concept: What Is Curriculum As It Applies To Medicalmentioning

confidence: 99%

Hospice and Palliative Medicine: Curriculum Evaluation and Learner Assessment in Medical Education

Sanchez-Reilly

Ross

2012

Journal of Palliative Medicine

View full text Add to dashboard Cite

Major efforts have been pursued to improve palliative care education for physicians at all levels of their training. Such changes include the incorporation of palliative care curriculum and guidelines, an established process for competency-based evaluation and certification, faculty development, innovative educational experiences, the improvement of textbooks, and the establishment of accredited palliative medicine fellowships. Hospice and palliative medicine (HPM) has been clearly defined as a subspecialty and a crucial area of medical education. As innovative curricular approaches have become available to educate medical and other interprofessional trainees, this article aims to describe different models and methods applied in curriculum evaluation, tailoring such approaches to the field of palliative medicine. A stepwise process of curriculum development and evaluation is described, focusing on available curriculum evaluation competency-based tools for each level of learners. As HPM evolves and its educational programs grow, curriculum evaluation will provides invaluable feedback to institutions and programs in many ways.

show abstract

“…Since 2013, the ACGME guidelines provide instructions to teach, train, and assess healthcare professionals within a competency‐based medical education (CBME) framework. Therefore, the need for authentic and complex assessments accurately reflecting the professional environment has since also been highlighted 1‐3 . It remains one of the current challenges in a CBME curriculum because of the several difficulties to conduct, organize, and develop valid assessments of the competences.…”

Section: Introductionmentioning

confidence: 99%

Usability and reproducibility of three tools to assess medical students and residents in emergency medicine

Philippon

Baud

Dumont

et al. 2021

AEM Education and Training

View full text Add to dashboard Cite

Objectives It is critical to assess competency of medical students and residents in emergency medicine (EM) during undergraduate and graduate medical education. However, very few valid tools exist to assess both technical and nontechnical skills in the specific context of EM. Three Acute Care Assessment Tools (ACAT 1, 2, and 3) have been previously developed for three acute care conditions: cardiac arrest (1), coma (2), and acute respiratory failure (3). This study aimed to evaluate the reproducibility of the tools. Methods The tool was tested using recorded videos of simulation sessions of fourth year medical students and first year residents in EM. Raters independently reviewed the videos two times in a 3‐month interval, and interrater and intrarater reliability using intraclass correlation (ICC) was calculated. Secondary endpoints included the completeness rate and relevance of the ACAT. Results Sixty‐two sessions were recorded and 48 videos analyzed (18 for CA and 15 for both respiratory failure and coma. The learners were residents in 32 (66%) of videos. Interrater reliability was excellent (ICC >0.9 for all three contexts) and so was the intrarater reliability (>0.88), both upon first review (month 0, M0) and at 3 months (M3). The usability of the ACAT was good, with a completeness of the items that ranged from 96% to 100%. Only one item of the ACAT 1 had a relevance of 27%, as it could not be completed in 13 scenarios out of 18. Conclusions The results demonstrate educators can evaluate students similarly utilizing video recordings of simulated medical scenario. The excellent completeness of the rated items advocated for good usability. The three ACATs can be utilized to assess for completeness of predefined tasks in three acute care broad scenario in a competency‐based medical education framework.

show abstract

How Should the ACGME Core Competencies Be Measured?

Cited by 3 publications

References 2 publications

Implementation and Evaluation of the Z-Score System for Normalizing Residency Evaluations

Implementation and Evaluation of the Z-Score System for Normalizing Residency Evaluations

Hospice and Palliative Medicine: Curriculum Evaluation and Learner Assessment in Medical Education

Usability and reproducibility of three tools to assess medical students and residents in emergency medicine

Contact Info

Product

Resources

About