Predicting Future Care Requirements Using Machine Learning for Pediatric Intensive and Routine Care Inpatients

Abstract

Develop and compare separate prediction models for ICU and non-ICU care for hospitalized children in four future time periods (6-12, 12-18, 18-24, and 24-30 hr) and assess these models in an independent cohort and simulated children's hospital. DESIGN:Predictive modeling used cohorts from the Health Facts database (Cerner Corporation, Kansas City, MO). SETTING:Children hospitalized in ICUs. PATIENTS:Children with greater than or equal to one ICU admission (n = 20,014) and randomly selected routine care children without ICU admission (n = 20,130) from 2009 to 2016 were used for model development and validation. An independent 2017-2018 cohort consisted of 80,089 children. INTERVENTIONS: None.MEASUREMENT AND MAIN RESULTS: Initially, we undersampled non-ICU patients for development and comparison of the models. We randomly assigned 64% of patients for training, 8% for validation, and 28% for testing in both clinical groups. Two additional validation cohorts were tested: a simulated children's hospitals and the 2017-2018 cohort. The main outcome was ICU care or non-ICU care in four future time periods based on physiology, therapy, and care intensity. Four independent, sequential, and fully connected neural networks were calibrated to risk of ICU care at each time period. Performance for all models in the test sample were comparable including sensitivity greater than or equal to 0.727, specificity greater than or equal to 0.885, accuracy greater than 0.850, area under the receiver operating characteristic curves greater than or equal to 0.917, and all had excellent calibration (all R 2 s > 0.98). Model performance in the 2017-2018 cohort was sensitivity greater than or equal to 0.545, specificity greater than or equal to 0.972, accuracy greater than or equal to 0.921, area under the receiver operating characteristic curves greater than or equal to 0.946, and R 2 s greater than or equal to 0.979. Performance metrics were comparable for the simulated children's hospital and for hospitals stratified by teaching status, bed numbers, and geographic location.CONCLUSIONS: Machine learning models using physiology, therapy, and care intensity predicting future care needs had promising performance metrics. Notably, performance metrics were similar as the prediction time periods increased from 6-12 hours to 24-30 hours.

Dynamic Mortality Risk Predictions for Children in ICUs: Development and Validation of Machine Learning Models*

“…This study demonstrated the applicability of the CI-M for assessments of serial changes in mortality risk for individuals. The Criticality Index was initially calibrated to probability of ICU care and has been applied to determining future care needs for hospitalized children (7). This study expands its use by recalibrating it to mortality risk.…”

“…We computed each admission's Criticality Index for each time period using the previously published machine learning methodology and detailed in Supplemental Digital Data 1 (http://links.lww.com/ PCC/B978) (5)(6)(7). Previously, we demonstrated that as the Criticality Index increases, the intensity and complexity of care increases.…”

“…The Criticality Index independent variables consist of six vital signs, 30 routinely measured laboratory variables, and medications classified into 143 categories and also included the number of measurements/ time period (5). The variables, definitions, and statistics have been reported in detail (5)(6)(7). Detailed data on the independent variables and imputed values are shown in Supplemental Digital Data 1 (http://links.…”

“…A recent "proof of concept" report showed potential for this approach (4). Recently, we developed the Criticality Index using machine learning, a new pediatric severity index for pediatric inpatients based on physiology, therapies, and intensity of care (5)(6)(7). This framework follows the prominent threads of critical care severity research including early qualitative assessments, identification and quantification of therapies, and physiologic profiles (8)(9)(10)(11)(12).…”

The criticality Index-mortality: A dynamic machine learning prediction algorithm for mortality prediction in children cared for in an ICU

“…Vital signs, laboratory data, and medication data were standardized to values from 0 to 1 using the maximum and minimum values of the training set. Consistent with other machine learning models, the data for each time period were forward imputed using the last available data if new data were not obtained ( 16 , 17 , 20 – 23 ) For the first time period, if vital signs or laboratory data were not obtained, we used the medians of the first time periods across all training patients adjusted by 9 age groups. The imputed values by age groups are reported in Supplementary Appendix S4 .…”