The ultrasound competency assessment tool for four-view cardiac POCUS

Abstract

Background Point-of-care ultrasound (POCUS) has been recognized as an essential skill across medicine. However, a lack of reliable and streamlined POCUS assessment tools with demonstrated validity remains a significant barrier to widespread clinical integration. The ultrasound competency assessment tool (UCAT) was derived to be a simple, entrustment-based competency assessment tool applicable to multiple POCUS applications. When used to assess a FAST, the UCAT demonstrated high internal consistency and moderate-to-excellent inter-rater reliability. The objective of this study was to validate the UCAT for assessment of a four-view transthoracic cardiac POCUS. Results Twenty-two trainees performed a four-view transthoracic cardiac POCUS in a simulated environment while being assessed by two observers. When used to assess a four-view cardiac POCUS the UCAT retained its high internal consistency ($$\alpha =0.90)$$ α = 0.90 ) and moderate-to-excellent inter-rater reliability (ICCs = 0.61–0.91; p’s ≤ 0.01) across all domains. The regression analysis suggestion that level of training, previous number of focused cardiac ultrasound, previous number of total scans, self-rated entrustment, and intent to pursue certification statistically significantly predicted UCAT entrustment scores [F (5,16) = 4.06, p = 0.01; R2 = 0.56]. Conclusion This study confirms the UCAT is a valid assessment tool for four-view transthoracic cardiac POCUS. The findings from this work and previous studies on the UCAT demonstrate the utility and flexibility of the UCAT tool across multiple POCUS applications and present a promising way forward for POCUS competency assessment.

Enhanced point of care ultrasound skills after additional instruction from simulated patients

“…Such use of OSCEs is consistent with previous published reports exploring the impact of SPteachers on student learning. 7-10 Each student's performance was rated using the Ultrasound Competency Assessment Tool 16,17 to ascertain a difference in performance between the two groups following the training session. This tool was chosen because it includes an entrustment score, making ratings easily interpretable through five basic anchors 18 compatible with competency based medical education in UGME.…”

External validation of the ultrasound competency assessment tool

“…Through direct observation, feedback, and structured evaluation, this tool allows supervisors to assess the trainee's ability to complete a task, ranging from total intervention to independence. Results from the UCAT study revealed excellent interrater reliability between two EM faculty observers as well as a high correlation between the domain assessment and entrustment scale 9,10 …”

“…In 2020, Bell et al 9,10 created the ultrasound competency assessment tool (UCAT) to address this gap. The UCAT is based on the results of a Delphi survey of Canadian emergency physicians and was validated by having EM residents perform a focused assessment with sonography in trauma (FAST) examination.…”

“…To ensure a baseline competency for residents before graduation from a training program, ultrasound competencybased assessment tools have been developed to gauge the skill level of trainees. 5,6 In 2020, Bell et al 9,10 created the ultrasound competency assessment tool (UCAT) to address this gap. The UCAT is based on the results of a Delphi survey of Canadian emergency physicians and was validated by having EM residents perform a focused assessment with sonography in trauma (FAST) examination.…”

“…Results from the UCAT study revealed excellent interrater reliability between two EM faculty observers as well as a high correlation between the domain assessment and entrustment scale. 9,10 The primary objective of our study was to externally validate the UCAT in a separate sample of EM residents and determine how the domains of the UCAT compare to an entrustment scale in isolation.…”