The Inter‐Rater Reliability of Pediatric Point‐of‐Care Lung Ultrasound Interpretation in Children With Acute Respiratory Failure

Abstract

Objectives-Use of point-of-care lung ultrasound (POC-LUS) has increased significantly in pediatrics yet it remains under-studied in the pediatric intensive care unit (PICU). No studies explicitly evaluate the reliability of POC-LUS artifact interpretation among critically ill children with acute respiratory failure (ARF) in the PICU. We thus designed this study to determine the inter-rater reliability of POC-LUS interpretation in pediatric ARF among pediatric intensivists trained in POC-LUS and an expert intensivist.Methods-We compared the interpretation of lung sliding, pleural line characteristics, ultrasound artifacts, and POC-LUS diagnoses among pediatric intensivists and an expert intensivist in a cohort of children admitted to the PICU for ARF. Kappa statistics (k) adjusted for maximum attainable agreement (k/k max ) were used to quantify chance-correct agreement between the pediatric intensivist and expert physician.Results-We enrolled 88 patients, evaluating 3 zones per hemithorax (anterior, lateral, and posterior) for lung sliding, pleural line characteristics, ultrasound artifacts, and diagnosis. There was moderate agreement between the PICU intensivist and expert-derived diagnoses with 56% observed agreement (k/k max = 0.46, 95% confidence interval [CI] 0.31-0.65). Agreement in identification of lung sliding (k = 0.19, 95% CI À0.17 to 0.56) and pleural line characteristics (k = 0.24, 95% CI 0.08-0.40) was slight and fair, respectively, while agreement in the interpretation of ultrasound artifacts ranged from moderate to substantial.Conclusions-Evidence supporting the evaluation of neonatal and adult patients with POC-LUS should not be extrapolated to critically ill pediatric patients. This study adds to the evidence supporting use of POC-LUS in the PICU by demonstrating moderate agreement between PICU intensivist and expert-derived POC-LUS diagnoses.

Artificial Intelligence Neural Network Consistently Interprets Lung Ultrasound Artifacts in Hospitalized Patients: A Prospective Observational Study

“…In spite of this unwanted scoring heterogeneity, point-of-care ultrasound is commonly used to inform patient care decisions. We also observed variability in our human scoring of lung ultrasound artifacts, furthermore the degree of variability was in line with existing evidence on this topic[7,18].…”

Inter-Rater Variability in the Evaluation of Lung Ultrasound in Videos Acquired from COVID-19 Patients

“…Our results are similar to other previous studies on inter-rater reliability for LUS outside of COVID-19. Previous investigations have demonstrated moderate to substantial agreement for B-lines [15][16][17], while this research shows only moderate to fair agreement for consolidations. This is similar to the results obtained in a previous LUS study with .…”

“…However, LUS is an operator-dependent imaging technique, and its utility depends on accurate acquisition and interpretation by bedside physicians [14][15][16][17]. Poor image acquisition and incorrect identification and interpretation of artifacts are potential sources of error in its clinical application [1].…”

Inter-rater Variability in the Evaluation of Lung Ultrasound on Videos Acquired from COVID-19 Patients

“…Our results are similar to other previous studies on interrater reliability for LUS outside of COVID-19. Previous investigations have demonstrated moderate to substantial agreement for B-lines [15][16][17]. In contrast, there is only moderate to a fair agreement for consolidations.…”

“…Furthermore, LUS reduces the risk of infection compared to other imaging modalities as these portable devices can be easily sanitized after the patient's examination [6,13]. However, LUS is an operator-dependent modality, and its utility depends on accurate acquisition and interpretation by bedside physicians [14][15][16][17]. Poor image acquisition and incorrect identification and interpretation of artifacts are potential sources of error in the clinical application of LUS [1].…”