The Assessment of Competency in Thoracic Sonography (ACTS) scale: validation of a tool for point-of-care ultrasound

Abstract

BackgroundThe rapid adoption of point-of-care ultrasound (POCUS) has created a need to develop assessment tools to ensure that learners can competently use these technologies. In this study, the authors developed and tested a rating scale to assess the quality of point-of-care thoracic ultrasound studies performed by novices. In Phase 1, the Assessment of Competency in Thoracic Sonography (ACTS) scale was developed based on structured interviews with subject matter experts. The tool was then piloted on a small series of ultrasound studies in Phase 2. In Phase 3 the tool was applied to a sample of 150 POCUS studies performed by ten learners; performance was then assessed by two independent raters.ResultsEvidence for the content validity of the ACTS scale was provided by a consensus exercise wherein experts agreed on the general principles and specific items that make up the scale. The tool demonstrated reasonable inter-rater reliability despite minimal requirements for evaluator training and displayed evidence of good internal structure, with related scale items correlating well with each other. Analysis of the aggregate learning curves suggested a rapid early improvement in learner performance with slower improvement after approximately 25–30 studies.ConclusionsThe ACTS scale provides a straightforward means to assess learner performance. Our results support the conclusion that the tool is an effective means of making valid judgments regarding competency in point-of-care thoracic ultrasound, and that the majority of learner improvement occurs during their first 25–30 practice studies.

Response

“…Many of those that do exist are based on fixed timeframes or number of examinations performed, neither of which is sufficient to ensure true competence, as seen time and again in other areas of clinical practice. 10 It is now the urgent responsibility of experts in this area to better define the means by which training in LUS, and the objective evaluation of the success of this same training occurs, 11,12 and incorporating this into widespread practice. The European Respiratory Society has recently launched an evidence-based training program based on these principles (see https://www.ersnet.org/professionaldevelopment/thoracic-ultrasound-training-programme), but there is still a long road to travel from this to routine implementation in everyday clinical practice.…”

Lung Ultrasound: The Emerging Role of Respiratory Therapists

“…The authors concluded that the scale was effective in making valid judgments regarding the competency of novices in point-of-care thoracic ultrasound, and the majority of learning improvement occurred in their first 25-30 practice studies. 101 Another publication on competency assessment of LUS was the LUS-Objective Structured Assessment of technical skills. The 26-element competency check includes 6 domains: indication for LUS examination, systematic approach, technical skills, recognition and differentiation of normal anatomy from pathology, documentation and reporting the examination, and diagnostic conclusion.…”

Technically Adequate Images for Preparticipation Screening Echocardiography Can Be Obtained by Novices After a Single Day of Training

“…Several previous studies examined the ability of novices to perform echocardiography after limited training and their ability to interpret basic pathologic findings, [9][10][11][12][13] whereas other studies specifically examined the technical quality of images produced by novices. [14][15][16][17][18] Most studies examined training in echocardiography within the hospital setting, but little has been done on the training requirements for echocardiography used out of hospitals as a screening tool for athletes.…”

“…The efficacy of a structured assessment of image quality in echocardiography has been highlighted in other studies evaluating the adequacy of images for conventional cardiology 22,23 as well as point of care emergency echocardiography. [16][17][18] Nair et al 22 compared the scores obtained by cardiology fellows in their in-training evaluation report, a nonstructured subjective assessment of scanning and interpretive skills, to the performance of the fellows in an objective structured clinical examination (OSCE). The OSCE was scored on the basis of a procedural checklist, which factored the technical aspects of the image and the measurements obtained.…”