Objectives: To validate the conceptual framework of “criticality,” a new pediatric inpatient severity measure based on physiology, therapy, and therapeutic intensity calibrated to care intensity, operationalized as ICU care. Design: Deep neural network analysis of a pediatric cohort from the Health Facts (Cerner Corporation, Kansas City, MO) national database. Setting: Hospitals with pediatric routine inpatient and ICU care. Patients: Children cared for in the ICU (n = 20,014) and in routine care units without an ICU admission (n = 20,130) from 2009 to 2016. All patients had laboratory, vital sign, and medication data. Interventions: None. Measurements and Main Results: A calibrated, deep neural network used physiology (laboratory tests and vital signs), therapy (medications), and therapeutic intensity (number of physiology tests and medications) to model care intensity, operationalized as ICU (versus routine) care every 6 hours of a patient’s hospital course. The probability of ICU care is termed the Criticality Index. First, the model demonstrated excellent separation of criticality distributions from a severity hierarchy of five patient groups: routine care, routine care for those who also received ICU care, transition from routine to ICU care, ICU care, and high-intensity ICU care. Second, model performance assessed with statistical metrics was excellent with an area under the curve for the receiver operating characteristic of 0.95 for 327,189 6-hour time periods, excellent calibration, sensitivity of 0.817, specificity of 0.892, accuracy of 0.866, and precision of 0.799. Third, the performance in individual patients with greater than one care designation indicated as 88.03% (95% CI, 87.72–88.34) of the Criticality Indices in the more intensive locations was higher than the less intense locations. Conclusions: The Criticality Index is a quantification of severity of illness for hospitalized children using physiology, therapy, and care intensity. This new conceptual model is applicable to clinical investigations and predicting future care needs.

Objectives: To assess severity of illness trajectories described by the Criticality Index for survivors and deaths in five patient groups defined by the sequence of patient care in ICU and routine patient care locations. Design: The Criticality Index developed using a calibrated, deep neural network, measures severity of illness using physiology, therapies, and therapeutic intensity. Criticality Index values in sequential 6-hour time periods described severity trajectories. Setting: Hospitals with pediatric inpatient and ICU care. Patients: Pediatric patients never cared for in an ICU (n = 20,091), patients only cared for in the ICU (n = 2,096) and patients cared for in both ICU and non-ICU care locations (n = 17,023) from 2009 to 2016 Health Facts database (Cerner Corporation, Kansas City, MO). Interventions: None. Measurements and Main Results: Criticality Index values were consistent with clinical experience. The median (25–75th percentile) ICU Criticality Index values (0.878 [0.696–0.966]) were more than 80-fold higher than the non-ICU values (0.010 [0.002–0.099]). Non-ICU Criticality Index values for patients transferred to the ICU were 40-fold higher than those never transferred to the ICU (0.164 vs 0.004). The median for ICU deaths was higher than ICU survivors (0.983 vs 0.875) (p < 0.001). The severity trajectories for the five groups met expectations based on clinical experience. Survivors had increasing Criticality Index values in non-ICU locations prior to ICU admission, decreasing Criticality Index values in the ICU, and decreasing Criticality Index values until hospital discharge. Deaths had higher Criticality Index values than survivors, steeper increases prior to the ICU, and worsening values in the ICU. Deaths had a variable course, especially those who died in non-ICU care locations, consistent with deaths associated with both active therapies and withdrawals/limitations of care. Conclusions: Severity trajectories measured by the Criticality Index showed strong validity, reflecting the expected clinical course for five diverse patient groups.

Develop and compare separate prediction models for ICU and non-ICU care for hospitalized children in four future time periods (6-12, 12-18, 18-24, and 24-30 hr) and assess these models in an independent cohort and simulated children's hospital. DESIGN:Predictive modeling used cohorts from the Health Facts database (Cerner Corporation, Kansas City, MO). SETTING:Children hospitalized in ICUs. PATIENTS:Children with greater than or equal to one ICU admission (n = 20,014) and randomly selected routine care children without ICU admission (n = 20,130) from 2009 to 2016 were used for model development and validation. An independent 2017-2018 cohort consisted of 80,089 children. INTERVENTIONS: None.MEASUREMENT AND MAIN RESULTS: Initially, we undersampled non-ICU patients for development and comparison of the models. We randomly assigned 64% of patients for training, 8% for validation, and 28% for testing in both clinical groups. Two additional validation cohorts were tested: a simulated children's hospitals and the 2017-2018 cohort. The main outcome was ICU care or non-ICU care in four future time periods based on physiology, therapy, and care intensity. Four independent, sequential, and fully connected neural networks were calibrated to risk of ICU care at each time period. Performance for all models in the test sample were comparable including sensitivity greater than or equal to 0.727, specificity greater than or equal to 0.885, accuracy greater than 0.850, area under the receiver operating characteristic curves greater than or equal to 0.917, and all had excellent calibration (all R 2 s > 0.98). Model performance in the 2017-2018 cohort was sensitivity greater than or equal to 0.545, specificity greater than or equal to 0.972, accuracy greater than or equal to 0.921, area under the receiver operating characteristic curves greater than or equal to 0.946, and R 2 s greater than or equal to 0.979. Performance metrics were comparable for the simulated children's hospital and for hospitals stratified by teaching status, bed numbers, and geographic location.CONCLUSIONS: Machine learning models using physiology, therapy, and care intensity predicting future care needs had promising performance metrics. Notably, performance metrics were similar as the prediction time periods increased from 6-12 hours to 24-30 hours.

Hospital mortality risk estimates determined at 6-hour time periods during care in the ICU. Models were truncated at 180 hours due to decreased sample size secondary to discharges and deaths. MEASUREMENTS AND MAIN RESULTS:The Criticality Index, based on physiology, therapy, and care intensity, was computed for each admission for each time period and calibrated to hospital mortality risk (Criticality Index-Mortality [CI-M]) at each of 29 time periods (initial assessment: 6 hr; last assessment: 180 hr). Performance metrics and clinical validity were determined from the held-out test sample (n = 3,453, 13%). Discrimination assessed with the area under the receiver operating characteristic curve was 0.852 (95% CI, 0.843-0.861) overall and greater than or equal to 0.80 for all individual time periods. Calibration assessed by the Hosmer-Lemeshow goodness-of-fit test showed good fit overall (p = 0.196) and was statistically not significant for 28 of the 29 time periods. Calibration plots for all models revealed the intercept ranged from--0.002 to 0.009, the slope ranged from 0.867 to 1.415, and the R 2 ranged from 0.862 to 0.989. Clinical validity assessed using population trajectories and changes in the risk status of admissions (clinical volatility) revealed clinical trajectories consistent with clinical expectations and greater clinical volatility in deaths than survivors (p < 0.001). CONCLUSIONS:Machine learning models incorporating physiology, therapy, and care intensity can track changes in hospital mortality risk during intensive care. The CI-M's framework and modeling method are potentially applicable to monitoring clinical improvement and deterioration in real time.

Objective: To examine medication administration records through electronic health record data to provide a broad description of the pharmaceutical exposure of critically ill children. Design: Retrospective cohort study using the Cerner Health Facts database. Setting: United States. Patients: A total of 43,374 children 7 days old to less than 22 years old receiving intensive care with available pharmacy data. Interventions: None. Measurements and Main Results: A total of 907,440 courses of 1,080 unique medications were prescribed with a median of nine medications (range, 1–99; 25–75th percentile, 5–16) per patient. The most common medications were acetaminophen, ondansetron, and morphine. Only 45 medications (4.2%) were prescribed to more than 5% of patients, and these accounted for 442,067 (48.7%) of the total courses of medications. Each additional medication was associated with increased univariate risk of mortality (odds ratio, 1.05; 95% CI, 1.05–1.06; p < 0.001). Conclusions: Children receiving intensive care receive a median of nine medications per patient and one quarter are prescribed at least than 16 medications. Only 45 medications were prescribed to more than 5% of patients, but these accounted for almost half of all medication courses.