Machine Learning–Based Critical Congenital Heart Disease Screening Using Dual‐Site Pulse Oximetry Measurements

Abstract

Background Oxygen saturation (Sp o 2 ) screening has not led to earlier detection of critical congenital heart disease (CCHD). Adding pulse oximetry features (ie, perfusion data and radiofemoral pulse delay) may improve CCHD detection, especially coarctation of the aorta (CoA). We developed and tested a machine learning (ML) pulse oximetry algorithm to enhance CCHD detection. Methods and Results Six sites prospectively enrolled newborns with and without CCHD and recorded simultaneous pre‐ and postductal pulse oximetry. We focused on models at 1 versus 2 time points and with/without pulse delay for our ML algorithms. The sensitivity, specificity, and area under the receiver operating characteristic curve were compared between the Sp o 2 ‐alone and ML algorithms. A total of 523 newborns were enrolled (no CHD, 317; CHD, 74; CCHD, 132, of whom 21 had isolated CoA). When applying the Sp o 2 ‐alone algorithm to all patients, 26.2% of CCHD would be missed. We narrowed the sample to patients with both 2 time point measurements and pulse‐delay data (no CHD, 65; CCHD, 14) to compare ML performance. Among these patients, sensitivity for CCHD detection increased with both the addition of pulse delay and a second time point. All ML models had 100% specificity. With a 2‐time‐points+pulse‐delay model, CCHD sensitivity increased to 92.86% ( P =0.25) compared with Sp o 2 alone (71.43%), and CoA increased to 66.67% ( P =0.5) from 0. The area under the receiver operating characteristic curve for CCHD and CoA detection significantly improved (0.96 versus 0.83 for CCHD, 0.83 versus 0.48 for CoA; both P =0.03) using the 2‐time‐points+pulse‐delay model compared with Sp o 2 alone. Conclusions ML pulse oximetry that combines oxygenation, perfusion data, and pulse delay at 2 time points may improve detection of CCHD and CoA within 48 hours after birth. Registration URL: https://www.clinicaltrials.gov/study/NCT04056104?term=NCT04056104&rank=1 ; Unique identifier: NCT04056104.

Principles of Artificial Intellgence for Medicine

“…At this point, the Siefkes et al. 's study 11 stems from a real clinically important problem demanding a solution, where authors select machine learning to solve this clinically important problem. Their solution mostly relies on data collected as part of standard of care with a minimal amount of added data points, pulse oximetry features, which increases its acceptability.…”

“…The analytical cohort in Siefkes et al. 's study 11 has a big advantage by prospectively collecting data from 6 sites, yet it had a small sample size with small event rate of CCHD, which was further lower for CoA outcome. Therefore, the potential of having data from multiple sites were not fully used as a low event rate prohibited authors to keep data from some of the sites as an external validation site.…”

“…In Siefkes et al. 's study, 11 the feature importance analysis was used as part of feature selection process, rather than to gain insight about the contribution of each predictor on the outcome. Therefore, the study could benefit from calculating Shapley additive explanations values 12 or implementing feature importance and direction analysis 4 , 13 to better understand the underlying mechanism identifying children with CCHD and CoA.…”

“… 10 In this issue of the Journal of the American Heart Association ( JAHA ), Siefkes et al. 11 propose a machine learning based CCHD screening using pulse oximetry measurements, beyond screening with SpO 2 alone.…”

“…In their study in this issue of JAHA , Siefkes et al. 11 investigate whether adding pulse oximetry features to SpO 2 and analyzing the data using machine learning methods improve the CCHD, especially coarctation of the Aorta (CoA). Authors prospectively collect data at 6 sites.…”