Machine-Learning vs. Expert-Opinion Driven Logistic Regression Modelling for Predicting 30-Day Unplanned Rehospitalisation in Preterm Babies: A Prospective, Population-Based Study (EPIPAGE 2)

Abstract

Introduction: Preterm babies are a vulnerable population that experience significant short and long-term morbidity. Rehospitalisations constitute an important, potentially modifiable adverse event in this population. Improving the ability of clinicians to identify those patients at the greatest risk of rehospitalisation has the potential to improve outcomes and reduce costs. Machine-learning algorithms can provide potentially advantageous methods of prediction compared to conventional approaches like logistic regression.Objective: To compare two machine-learning methods (least absolute shrinkage and selection operator (LASSO) and random forest) to expert-opinion driven logistic regression modelling for predicting unplanned rehospitalisation within 30 days in a large French cohort of preterm babies.Design, Setting and Participants: This study used data derived exclusively from the population-based prospective cohort study of French preterm babies, EPIPAGE 2. Only those babies discharged home alive and whose parents completed the 1-year survey were eligible for inclusion in our study. All predictive models used a binary outcome, denoting a baby's status for an unplanned rehospitalisation within 30 days of discharge. Predictors included those quantifying clinical, treatment, maternal and socio-demographic factors. The predictive abilities of models constructed using LASSO and random forest algorithms were compared with a traditional logistic regression model. The logistic regression model comprised 10 predictors, selected by expert clinicians, while the LASSO and random forest included 75 predictors. Performance measures were derived using 10-fold cross-validation. Performance was quantified using area under the receiver operator characteristic curve, sensitivity, specificity, Tjur's coefficient of determination and calibration measures.Results: The rate of 30-day unplanned rehospitalisation in the eligible population used to construct the models was 9.1% (95% CI 8.2–10.1) (350/3,841). The random forest model demonstrated both an improved AUROC (0.65; 95% CI 0.59–0.7; p = 0.03) and specificity vs. logistic regression (AUROC 0.57; 95% CI 0.51–0.62, p = 0.04). The LASSO performed similarly (AUROC 0.59; 95% CI 0.53–0.65; p = 0.68) to logistic regression.Conclusions: Compared to an expert-specified logistic regression model, random forest offered improved prediction of 30-day unplanned rehospitalisation in preterm babies. However, all models offered relatively low levels of predictive ability, regardless of modelling method.

The past, current, and future of neonatal intensive care units with artificial intelligence: a systematic review

“… - Low prediction rate with ML Podda et al, 2018 66 ANN Development of the Preterm Infants Survival Assessment (PISA) predictor Between 2008 and 2014, 23747 neonates (<30 weeks gestational age or <1501 g birth weight were recruited Italian Neonatal Network 12 easily collected perinatal variables 91.3% (AUC) 77.9% (AUC) 82.8% (AUC) 88.6% (AUC) + NN had a slightly better discrimination than logistic regression - Like all other model-based methods, is still too imprecise to be used for predicting an individual infant’s outcome - Retrospective design - Lack of variables Turova et al, 2020 85 Random Forest To predict intraventricular hemorrhage in 23–30 weeks of GA infants 229 infants Clinical variables and cerebral blood flow (extracted from mathematical calculation) were used 10 fold validation 86%–93% (AUC) Vary on the extracted features in and feature weight in the model + Good accuracy - Retrospective - Gender distribution was not standardized between the groups - Not corresponding lab value according to the IVH time Cabrera-Quiros et al, 2021 145 Logistic regressor, naive Bayes, and nearest mean classifier Prediction of late-onset sepsis (starting after the third day of life) in preterm babies based on various patient monitoring data 24 hours before onset 32 premature infants with sepsis and 32 age-matched control patients Heart rate variability, respiration, and body motion, differences between late-onset sepsis and Control group were visible up to 5 hours preceding the cultures, resuscitation, and antibiotics started here (CRASH) point Combination of all features showed a mean accuracy 79% and mean precision rate 82% 3 hours before the onset of sepsis Naive Bayes accuracy: 71% Nearest Mean: 70% + Monitoring of vital parameters could be predicted late onset sepsis up to 5 hours. - Small sample size - Retrospective - Gestational age, postnatal age, sepsis and culture Reed et al, 2021 143 Comparison least absolute shrinkage and selection operator (LASSO) and random forest (RF) to expert-opinion driven logistic regression modeling Prediction of 30-day unplanned rehospitalization of preterm babies 5567 live-born babies and 3841 were included to the study Data derived exclusively from The...…”

“…EHR and medical records were featured in ML algorithms for the diagnosis of congenital heart defects 135 , HIE (Hypoxic Ischemic Encephalopathy) 136 , IVH (Intraventricular Hemorrhage) 84 , 85 , neonatal jaundice 137 , 138 , prediction of NEC (Necrotizing Enterocolitis) 139 , prediction of neurodevelopmental outcome in ELBW (extremely low birth weight) infants 65 , 140 , 141 , prediction of neonatal surgical site infections 142 , and prediction of rehospitalization 143 (Table 5 ).…”

Machine learning methods to predict attrition in a population-based cohort of very preterm infants

“…A superior performance of Random Forest over Logistic Regression for predictive models was shown in diverse biomedical applications, such as suicidal behaviour 36 , cancer metastasis 37 , readmissions in patients with heart failure 38 and, unplanned rehospitalisation of preterm babies 39 . Likewise, a massive experimental evaluation of 179 algorithms using 121 datasets showed that Random Forest was very close to the best attainable accuracy for most of the datasets 40 .…”