Validation of a machine learning algorithm for early severe sepsis prediction: a retrospective study predicting severe sepsis up to 48 h in advance using a diverse dataset from 461 US hospitals

Abstract

Background Severe sepsis and septic shock are among the leading causes of death in the United States and sepsis remains one of the most expensive conditions to diagnose and treat. Accurate early diagnosis and treatment can reduce the risk of adverse patient outcomes, but the efficacy of traditional rule-based screening methods is limited. The purpose of this study was to develop and validate a machine learning algorithm (MLA) for severe sepsis prediction up to 48 h before onset using a diverse patient dataset. Methods Retrospective analysis was performed on datasets composed of de-identified electronic health records collected between 2001 and 2017, including 510,497 inpatient and emergency encounters from 461 health centers collected between 2001 and 2015, and 20,647 inpatient and emergency encounters collected in 2017 from a community hospital. MLA performance was compared to commonly used disease severity scoring systems and was evaluated at 0, 4, 6, 12, 24, and 48 h prior to severe sepsis onset. Results 270,438 patients were included in analysis. At time of onset, the MLA demonstrated an AUROC of 0.931 (95% CI 0.914, 0.948) and a diagnostic odds ratio (DOR) of 53.105 on a testing dataset, exceeding MEWS (0.725, P < .001; DOR 4.358), SOFA (0.716; P < .001; DOR 3.720), and SIRS (0.655; P < .001; DOR 3.290). For prediction 48 h prior to onset, the MLA achieved an AUROC of 0.827 (95% CI 0.806, 0.848) on a testing dataset. On an external validation dataset, the MLA achieved an AUROC of 0.948 (95% CI 0.942, 0.954) at the time of onset, and 0.752 at 48 h prior to onset. Conclusions The MLA accurately predicts severe sepsis onset up to 48 h in advance using only readily available vital signs extracted from the existing patient electronic health records. Relevant implications for clinical practice include improved patient outcomes from early severe sepsis detection and treatment.

Machine Learning and Syncope Management in the ED: The Future Is Coming

“…Mao et al [45] validated an ML algorithm with gradient tree boosting, InSight, providing high sensitivity and specificity for the detection and prediction of sepsis, severe sepsis, and septic shock using the analysis of only six common vital signs taken from EHRs (i.e., systolic blood pressure, diastolic blood pressure, heart rate, respiratory rate, peripheral capillary oxygen saturation and temperature). Similar results were confirmed by other authors [46,47]. Other areas in which ML models were successfully applied to diagnostic decision making include influenza [48,49], urinary tract infections [50], chronic obstructive pulmonary disease and asthma exacerbations [51], myocardial infarction [52], appendicitis [53,54].…”

Trends in US Pediatric Hospital Admissions in 2020 Compared With the Decade Before the COVID-19 Pandemic

“…As complexity in medical information has increased, there has been progressive interest in the application of machine learning to facilitate clinical decision-making. Such algorithms have been studied in a diverse array of applications, including sepsis prediction, 52 COVID-19 prognosis, 53 population health, 54 and cyberbullying. 55 In the present study, before the COVID-19 pandemic, 6 , 7 ensemble time-series forecasting models accurately predicted 70 of 72 monthly pediatric admission rates (97.2%) between July 2019 and December 2019.…”

Immune Deregulation in Sepsis and Septic Shock: Reversing Immune Paralysis by Targeting PD-1/PD-L1 Pathway

“…It has been shown that an algorithm constructed by machine learning can recognize sepsis hours earlier than can be done by humans in clinical practice ( 121 ). In fact, an algorithm created by AI that can predict sepsis up to 48 h in advance has been reported ( 122 ). Achieving early diagnoses using multiple parameters is a capability unique to the innumerable calculations made possible by machines.…”