Evaluating a sepsis prediction machine learning algorithm in the emergency department and intensive care unit: a before and after comparative study

Burdick, Hoyt; Pino, Eduardo; Gabel-Comeau, Denise; Gu, Carol; Huang, Hui‐Bin; Lynn-Palevsky, Anna; Das, Ritankar

doi:10.1101/224014

Cited by 14 publications

(14 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Use of the previous algorithm was also associated with earlier administration of antibiotics and ordering of blood cultures [30]. The previous algorithm has also improved sepsis-related outcomes at Cape Regional Medical Center [31] and Cabell Huntington Hospital [32], and has been successfully integrated with various EHRs including Epic, Cerner, McKesson, Meditech, Allscripts, Soarian, and Vista. While the present study examines only the UCSF and BIDMC datasets, these prior results are consistent with a broad applicability of MLAs to a variety of datasets and clinical data collection methods.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Evaluation of a machine learning algorithm for up to 48-hour advance prediction of sepsis using six vital signs

Barton

Chettipally

Zhou

et al. 2019

Computers in Biology and Medicine

Self Cite

114

View full text Add to dashboard Cite

Objective: Sepsis remains a costly and prevalent syndrome in hospitals; however, machine learning systems can increase timely sepsis detection using electronic health records. This study validates a gradient boosted ensemble machine learning tool for sepsis detection and prediction, and compares its performance to existing methods. Materials and Methods: Retrospective data was drawn from databases at the University of California, San Francisco (UCSF) Medical Center and the Beth Israel Deaconess Medical Center (BIDMC). Adult patient encounters without sepsis on admission, and with at least one recording of each of six vital signs (SpO 2 , heart rate, respiratory rate, temperature, systolic and diastolic blood pressure) were included. We compared the performance of the machine learning algorithm (MLA) to that of commonly used scoring systems. Area under the receiver operating characteristic (AUROC) curve was our primary measure of accuracy. MLA performance was measured at sepsis onset and 24, and 48 hours prior sepsis onset. Results: The MLA achieved AUROC of 0.88, 0.84, and 0.83 for sepsis onset and 24 and 48 hours prior to onset, respectively. These values were superior to those of SIRS (0.66), MEWS (0.61), SOFA (0.72), and qSOFA (0.60) at time of onset. When trained on UCSF data and tested on BIDMC data, sepsis onset AUROC was 0.89.

show abstract

Section: Discussionmentioning

confidence: 99%

“…The sepsis gold standard used in this study is necessarily an imperfect characterization of sepsis. We nevertheless believe it to be useful in developing a sepsis prediction tool, as demonstrated by the improvements in sepsis-related clinical outcomes seen using sepsis prediction algorithms trained with the same gold standard [31,32].…”

Section: Limitationsmentioning

confidence: 99%

Evaluation of a machine learning algorithm for up to 48-hour advance prediction of sepsis using six vital signs

Barton

Chettipally

Zhou

et al. 2019

Computers in Biology and Medicine

Self Cite

114

View full text Add to dashboard Cite

show abstract

“…Shimabukuro and colleagues [14] employed machine learning techniques to clinical records of 142 patients with severe sepsis from University of California San Francisco Medical Center (California, USA) to predict the in-hospital length of stay and mortality rate. Burdick et al [15] used several computational intelligence methods on medical records of 2,296 patients related to sepsis, that were provided by Cabell Huntington Hospital (Huntington, West Virginia, USA). Their goal was to predict patients' mortality and in-hospital length of stay.…”

Section: Introductionmentioning

confidence: 99%

Data analytics and clinical feature ranking of medical records of patients with sepsis

Chicco

Oneto

2021

BioData Mining

View full text Add to dashboard Cite

Background Sepsis is a life-threatening clinical condition that happens when the patient’s body has an excessive reaction to an infection, and should be treated in one hour. Due to the urgency of sepsis, doctors and physicians often do not have enough time to perform laboratory tests and analyses to help them forecast the consequences of the sepsis episode. In this context, machine learning can provide a fast computational prediction of sepsis severity, patient survival, and sequential organ failure by just analyzing the electronic health records of the patients. Also, machine learning can be employed to understand which features in the medical records are more predictive of sepsis severity, of patient survival, and of sequential organ failure in a fast and non-invasive way. Dataset and methods In this study, we analyzed a dataset of electronic health records of 364 patients collected between 2014 and 2016. The medical record of each patient has 29 clinical features, and includes a binary value for survival, a binary value for septic shock, and a numerical value for the sequential organ failure assessment (SOFA) score. We disjointly utilized each of these three factors as an independent target, and employed several machine learning methods to predict it (binary classifiers for survival and septic shock, and regression analysis for the SOFA score). Afterwards, we used a data mining approach to identify the most important dataset features in relation to each of the three targets separately, and compared these results with the results achieved through a standard biostatistics approach. Results and conclusions Our results showed that machine learning can be employed efficiently to predict septic shock, SOFA score, and survival of patients diagnoses with sepsis, from their electronic health records data. And regarding clinical feature ranking, our results showed that Random Forests feature selection identified several unexpected symptoms and clinical components as relevant for septic shock, SOFA score, and survival. These discoveries can help doctors and physicians in understanding and predicting septic shock. We made the analyzed dataset and our developed software code publicly available online.

show abstract

“…35 36 Machine learningbased decision support systems, therefore, represent an important area of investigation for sepsis research. 37 38 The MLA used in this study has been described in previous peer-reviewed publications both retrospectively and prospectively, [39][40][41][42][43][44][45] but has not been evaluated for its effect on clinical outcomes on multicentre diverse hospital settings. In this study, performance of our MLA for severe sepsis prediction and detection was evaluated using real-world data from patient EHRs at nine diverse hospitals from the northeast, southern, midwestern and western regions of USA, spanning academic centres to community hospitals.…”

Section: Introductionmentioning

confidence: 99%

Effect of a sepsis prediction algorithm on patient mortality, length of stay and readmission: a prospective multicentre clinical outcomes evaluation of real-world patient data from US hospitals

Burdick

Pino

Gabel-Comeau

et al. 2020

BMJ Health Care Inform

Self Cite

View full text Add to dashboard Cite

BackgroundSevere sepsis and septic shock are among the leading causes of death in the USA. While early prediction of severe sepsis can reduce adverse patient outcomes, sepsis remains one of the most expensive conditions to diagnose and treat.ObjectiveThe purpose of this study was to evaluate the effect of a machine learning algorithm for severe sepsis prediction on in-hospital mortality, hospital length of stay and 30-day readmission.DesignProspective clinical outcomes evaluation.SettingEvaluation was performed on a multiyear, multicentre clinical data set of real-world data containing 75 147 patient encounters from nine hospitals across the continental USA, ranging from community hospitals to large academic medical centres.ParticipantsAnalyses were performed for 17 758 adult patients who met two or more systemic inflammatory response syndrome criteria at any point during their stay (‘sepsis-related’ patients).InterventionsMachine learning algorithm for severe sepsis prediction.Outcome measuresIn-hospital mortality, length of stay and 30-day readmission rates.ResultsHospitals saw an average 39.5% reduction of in-hospital mortality, a 32.3% reduction in hospital length of stay and a 22.7% reduction in 30-day readmission rate for sepsis-related patient stays when using the machine learning algorithm in clinical outcomes analysis.ConclusionsReductions of in-hospital mortality, hospital length of stay and 30-day readmissions were observed in real-world clinical use of the machine learning-based algorithm. The predictive algorithm may be successfully used to improve sepsis-related outcomes in live clinical settings.Trial registration numberNCT03960203

show abstract

Evaluating a sepsis prediction machine learning algorithm in the emergency department and intensive care unit: a before and after comparative study

Abstract: Introduction Sepsis is a major health crisis in US hospitals, and several clinical identification

Cited by 14 publications

References 23 publications

Evaluation of a machine learning algorithm for up to 48-hour advance prediction of sepsis using six vital signs

Evaluation of a machine learning algorithm for up to 48-hour advance prediction of sepsis using six vital signs

Data analytics and clinical feature ranking of medical records of patients with sepsis

Effect of a sepsis prediction algorithm on patient mortality, length of stay and readmission: a prospective multicentre clinical outcomes evaluation of real-world patient data from US hospitals

Contact Info

Product

Resources

About