The Ultrasound Competency Assessment Tool (UCAT): Development and Evaluation of a Novel Competency‐based Assessment Tool for Point‐of‐care Ultrasound

Objectives There has been a steady increase in the growth and utilization of point‐of‐care ultrasound (POCUS) in pediatric emergency medicine (PEM). POCUS has been established as an Accreditation Council for Graduate Medical Education (ACGME) core requirement for accreditation of PEM fellowship programs. Despite this requirement, training guidelines regarding POCUS knowledge and skills have yet to be developed. The purpose of this project was to develop a curriculum and a competency checklist for PEM fellow POCUS education. Methods We formed a core leadership group based on expertise in one or more key areas: PEM, POCUS, curriculum development, or Delphi methods. We recruited 29 PEM POCUS or ultrasound education experts from North America to participate in a three‐round electronic Delphi project. The first Delphi round asked experts to generate a list of the core POCUS knowledge and skills that a PEM fellow would need during training to function as an autonomous practitioner. Subsequent rounds prioritized the list of knowledge and skills, and the core leadership group organized knowledge and skills into global competencies and subcompetencies. Results The first Delphi round yielded 61 POCUS areas of knowledge and skills considered important for PEM fellow learning. After two subsequent Delphi rounds, the list of POCUS knowledge and skills was narrowed to 38 items that addressed elements of six global competencies. The core leadership group then revised items into subcompetencies and categorized them under global competencies, developing a curriculum that defined the scope (depth of content) and sequence (order of teaching) of these POCUS knowledge and skill items. Conclusions This expert, consensus‐generated POCUS curriculum provides detailed guidance for PEM fellowships to incorporate POCUS education into their programs. Our curriculum also identifies core ultrasound knowledge and skills needed by PEM fellows to perform the specific POCUS applications recommended in prior publications.

Section: Discussionmentioning

confidence: 94%

Design of a point‐of‐care ultrasound curriculum for pediatric emergency medicine fellows: A Delphi study

Gold

Levine

Hsu

et al. 2021

“…In recent years there has been important work developing and collecting validity evidence for POCUS assessment tools 15–19 . These tools are designed to assess the technical skills of image acquisition and optimization but require a human assessor to be present 15 . This may be feasible at some institutions; however, it could limit generalizability to the global POCUS community.…”

Section: Discussionmentioning

confidence: 99%

“…This may be feasible at some institutions; however, it could limit generalizability to the global POCUS community. The interrater reliability for these tools are also variable 15 . The concept of integrating sensors, specifically hand‐motion analysis, into a comprehensive and evidence‐based POCUS assessment tools has previously been suggested 11 .…”

Section: Discussionmentioning

confidence: 99%

“…The blinded, anonymized, videos were later analyzed by two independent POCUS experts. Two of the four domains (image acquisition and image optimization) of the Ultrasound Competency Assessment Tool (UCAT), 15 a previously published ultrasound competence assessment tool, were used to grade the proficiency of the scan (Table S4). Modification of the UCAT tool was necessary as domains “preparation” and “clinical integration” were not assessed.…”

Section: Methodsmentioning

confidence: 99%

“…[15][16][17][18][19] These tools are designed to assess the technical skills of image acquisition and optimization but require a human assessor to be present. 15 This may be feasible at some institutions; however, it could limit generalizability to the global POCUS community. The interrater reliability for these tools are also variable.…”

Section: Sensor Datamentioning

confidence: 99%

See 2 more Smart Citations

Characterizing the biomechanical differences between novice and expert point‐of‐care ultrasound practitioners using a low‐cost gyroscope and accelerometer integrated sensor: A pilot study

Prager

Pageau

Hodges

et al. 2022

Introduction: Point-of-care ultrasound (POCUS) has become an important diagnostic tool in acute care medicine; however, little is known about the biomechanical differences between novice and expert practitioners. Methods:A low-cost ($50 CAD) gyroscope and accelerometer integrated sensor was assembled and affixed to an ultrasound probe. Seventeen participants, nine novices and eight experts, were recruited to perform three abdominal and four cardiac scans on a standardized patient. Participant demographics, time per scan, average acceleration, average angular velocity, decay in acceleration and angular velocity over time, and frequency of probe movements were analyzed. Video capture with blinded video review was scored.Results: On video review, experts had higher image optimization and acquisition scores for both abdominal and cardiac scans. Experts had shorter scan times for abdominal (7 s vs. 26 s, p = 0.003) and cardiac (11 s vs. 26 s, p < 0.001) scans. There was no difference in average acceleration (g) between novices and experts performing abdominal (1.02 vs. 1.01, p = 0.50) and cardiac (1.01 vs. 1.01, p = 0.45) scans. Experts had lower angular velocity (°/s) for abdominal scans (10.00 vs. 18.73, p < 0.001) and cardiac scans (15.61 vs. 20.33, p = 0.02) There was a greater decay in acceleration over time for experts performing cardiac scans compared to novices (−0.194 vs. −0.050, p = 0.03) but not for abdominal scans or when measuring angular velocity.The frequency of movements (Hz) was higher for novices compared to experts for abdominal (16.68 vs. 13.79, p < 0.001) and cardiac (17.60 vs. 13.63, p = 0.002) scans.Discussion: This study supports the feasibility of a low-cost gyroscope and accelerometer integrated sensor to quantify the biomechanics of POCUS. It may also support the concept of "window shopping" as a method by which experts obtain abdominal

Trends in point‐of‐care ultrasound use among emergency medicine residency programs over a 10‐year period

Gottlieb

Cooney

King

et al. 2023

Background Point‐of‐care ultrasound (POCUS) is increasingly utilized in emergency medicine (EM). While residents are required by the Accreditation Council for General Medical Education to complete a minimum of 150 POCUS examinations before graduation, the distribution of examination types is not well‐described. This study sought to assess the number and distribution of POCUS examinations completed during EM residency training and evaluate trends over time. Methods This was a 10‐year retrospective review of POCUS examinations across five EM residency programs. The study sites were deliberately selected to represent diversity in program type, program length, and geography. Data from EM residents graduating from 2013 to 2022 were eligible for inclusion. Exclusion criteria were residents in combined training programs, residents who did not complete all training at one institution, and residents who did not have data available. Examination types were identified from the American College of Emergency Physicians guidelines for POCUS. Each site obtained POCUS examination totals for every resident upon graduation. We calculated the mean and 95% confidence interval for each procedure across study years. Results A total of 535 residents were eligible for inclusion, with 524 (97.9%) meeting all inclusion criteria. The mean number of POCUS examinations per resident increased by 46.9% from 277 in 2013 to 407 in 2022. All examination types had stable or increasing frequency. Focused assessment with sonography in trauma (FAST), cardiac, obstetric/gynecologic, and renal/bladder were performed most frequently. Ocular, deep venous thrombosis, musculoskeletal, skin/soft tissue, thoracic, and cardiac examinations had the largest percentage increase in numbers over the 10‐year period, while bowel and testicular POCUS remained rare. Conclusions There was an overall increase in the number of POCUS examinations performed by EM residents over the past 10 years, with FAST, cardiac, obstetric/gynecologic, and renal/bladder being the most common examination types. Among less common procedures, increased frequency may be needed to ensure competence and avoid skill decay for those examination types. This information can help inform POCUS training in residency and accreditation requirements.