Multicenter validation of a sepsis prediction algorithm using only vital sign data in the emergency department, general ward and ICU

Abstract

Objectives: We validate a machine learning-based sepsis prediction algorithm (InSight) for detection and prediction of three sepsis-related gold standards, using only six vital signs. We evaluate robustness to missing data, customization to site-specific data using transfer learning, and generalizability to new settings. Design:A machine learning algorithm with gradient tree boosting. Features for prediction were created from combinations of only six vital sign measurements and their changes over time. Conclusions:InSight outperforms existing sepsis scoring systems in identifying and predicting sepsis, severe sepsis, and septic shock. This is the first sepsis screening system to exceed an AUROC of 0.90 using only vital sign inputs. InSight is robust to missing data, can be customized to novel hospital data using a small fraction of site data, and retained strong discrimination across all institutions. Strengths and limitations of this study •Machine learning is applied to the detection and prediction of three separate sepsis standards in the emergency department, general ward and intensive care settings.

Validation of a machine learning algorithm for early severe sepsis prediction: a retrospective study predicting severe sepsis up to 48 h in advance using a diverse dataset from 461 US hospitals

“…This dataset includes patient data from intensive care unit and floor wards, representing a variety of data collection frequencies and care provision levels. This dataset is significantly larger and more diverse than datasets used to develop previous versions of the algorithm, which has been applied to sepsis and severe sepsis detection using only vital sign data in the emergency department, general ward and ICU [ 37 , 40 – 42 ] and has been evaluated for its effect on clinical outcomes in a single-center study [ 39 ] as well as a randomised clinical trial [ 38 ].…”

“…Performance metrics of the algorithm were evaluated and compared against common rule-based methods using retrospective patient data from 461 hospitals and an external validation data set from Cabell Huntington Hospital. To address the growing need for rigorous external validation on diverse datasets [ 35 ], this algorithm was developed and evaluated on significantly larger and more diverse datasets than previously investigated [ 37 – 43 ].…”

Practical Machine Learning-Based Sepsis Prediction

“…Recently, an ML algorithm using only 6 vital signs: systolic blood pressure, diastolic blood pressure, heart rate, respiratory rate, peripheral capillary oxygen saturation, and temperature obtained in the emergency department, general ward, and ICU has shown to outperform current scoring systems: SOFA, SIRS and modified early warning score systems for the detection and prediction of sepsis and septic shock [15]. As more reliable and complete datasets of continuous vital sign monitoring become available, systems using only objective features will likely improve.…”

Evaluating a sepsis prediction machine learning algorithm in the emergency department and intensive care unit: a before and after comparative study

“…Risk prediction scores were computed hourly throughout the duration of each patient's stay. The MLA used in this study is described in detail in prior prospective [16,17] and retrospective work [13].…”

“…West Virginia provider Cabell Huntington Hospital (CHH), a 303-bed facility, partnered with Dascena (Hayward, CA) to improve sepsis-related outcomes using a machine learning algodiagnostic (MLA). The Dascena MLA was validated for sepsis prediction and detection in several studies [13][14][15], demonstrating an area under the receiver operator characteristic (ROC) curve (AUROC) over 0.90 using only six vital signs, in a multicenter cohort study of over 650,000 encounters [16]. In a recent randomized clinical trial, mortality decreased by 12.4 percentage points with use of the MLA, a relative reduction of 58% [17].…”