Evaluation of a machine learning algorithm for up to 48-hour advance prediction of sepsis using six vital signs

Barton, Christopher; Chettipally, Uli K.; Zhou, Yifan; Jiang, Zirui; Lynn-Palevsky, Anna; Le, Sidney; Calvert, Jacob; Das, Ritankar

doi:10.1016/j.compbiomed.2019.04.027

Cited by 114 publications

(84 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Most of the studies were carried out in the ICU (n = 15; 54%), followed by hospital wards (n = 7; 25%) and the emergency department (ED, n = 4; 14%). Two studies by Barton et al, and Mao et al, examined all of these settings [25,27]. In the intensive care, most of the studies modeled sepsis as their target condition (n = 10; 67%), compared to severe sepsis (n = 3; 20%) or septic shock (n = 2; 13%).…”

Section: Study Characteristicsmentioning

confidence: 99%

“…It should be noted that many models were developed on similar populations. Specifically, numerous models were tested on the freely accessible MIMIC database [27,33,[52][53][54][55][56][57][58][59] and all models were developed in the United States. The current trend holds risks for promoting inequality in healthcare as no models were developed or validates in middle or low income countries.…”

Section: Future Directions and Academic Contributionmentioning

confidence: 99%

See 1 more Smart Citation

Machine learning for the prediction of sepsis: a systematic review and meta-analysis of diagnostic test accuracy

et al. 2020

View full text Add to dashboard Cite

Purpose: Early clinical recognition of sepsis can be challenging. With the advancement of machine learning, promising real-time models to predict sepsis have emerged. We assessed their performance by carrying out a systematic review and meta-analysis. Methods: A systematic search was performed in PubMed, Embase.com and Scopus. Studies targeting sepsis, severe sepsis or septic shock in any hospital setting were eligible for inclusion. The index test was any supervised machine learning model for real-time prediction of these conditions. Quality of evidence was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology, with a tailored Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) checklist to evaluate risk of bias. Models with a reported area under the curve of the receiver operating characteristic (AUROC) metric were meta-analyzed to identify strongest contributors to model performance. Results: After screening, a total of 28 papers were eligible for synthesis, from which 130 models were extracted. The majority of papers were developed in the intensive care unit (ICU, n = 15; 54%), followed by hospital wards (n = 7; 25%), the emergency department (ED, n = 4; 14%) and all of these settings (n = 2; 7%). For the prediction of sepsis, diagnostic test accuracy assessed by the AUROC ranged from 0.68-0.99 in the ICU, to 0.96-0.98 in-hospital and 0.87 to 0.97 in the ED. Varying sepsis definitions limit pooling of the performance across studies. Only three papers clinically implemented models with mixed results. In the multivariate analysis, temperature, lab values, and model type contributed most to model performance. Conclusion: This systematic review and meta-analysis show that on retrospective data, individual machine learning models can accurately predict sepsis onset ahead of time. Although they present alternatives to traditional scoring systems, between-study heterogeneity limits the assessment of pooled results. Systematic reporting and clinical implementation studies are needed to bridge the gap between bytes and bedside.

show abstract

Section: Study Characteristicsmentioning

confidence: 99%

Section: Future Directions and Academic Contributionmentioning

confidence: 99%

Machine learning for the prediction of sepsis: a systematic review and meta-analysis of diagnostic test accuracy

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Specifically, because of the non-linear nature of its algorithm, its accuracy in predicting postoperative sepsis has been shown to be several folds superior in this patient population [95]. In addition to POTTER, machine learning techniques have been used to analyze vital signs and heart rate variability in real time to predict patients in early sepsis [96][97][98][99][100][101]. For example, the machine learning algorithm Insight was derived from six vital signs and outperformed existing scoring systems for sepsis and septic shock [99].…”

Section: Predicting Sepsismentioning

confidence: 99%

Sepsis 2019: What Surgeons Need to Know

Kaafarani

Rattan

et al. 2020

Surgical Infections

View full text Add to dashboard Cite

The definition of sepsis continues to be as dynamic as the management strategies used to treat this. Sepsis-3 has replaced the earlier systemic inflammatory response syndrome (SIRS)-based diagnoses with the rapid Sequential Organ Failure Assessment (SOFA) score assisting in predicting overall prognosis with regards to mortality. Surgeons have an important role in ensuring adequate source control while recognizing the threat of carbapenem-resistance in gram-negative organisms. Rapid diagnostic tests are being used increasingly for the early identification of multi-drug-resistant organisms (MDROs), with a key emphasis on the multidisciplinary alert of results. Novel, higher generation antibiotic agents have been developed for resistance in ESKCAPE (Enterococcus faecium, Staphylococcus aureus, Klebsiella pneumoniae, Acinetobacter baumannii, Pseudomonas aeruginosa, and Enterobacter species) organisms while surgeons have an important role in the prevention of spread. The Study to Optimize Peritoneal Infection Therapy (STOP-IT) trial has challenged the previous paradigm of length of antibiotic treatment whereas biomarkers such as procalcitonin are playing a prominent role in individualizing therapy. Several novel therapies for refractory septic shock, while still investigational, are gaining prominence rapidly (such as vitamin C) whereas others await further clinical trials. Management strategies presented as care bundles continue to be updated by the Surviving Sepsis Campaign, yet still remain controversial in its global adoption. We have broadened our temporal and epidemiologic perspective of sepsis by understanding it both as an acute, time-sensitive, life-threatening illness to a chronic condition that increases the risk of mortality up to five years post-discharge. Artificial intelligence, machine learning, and bedside scoring systems can assist the clinician in predicting post-operative sepsis. The public health role of the surgeon is key. This includes collaboration and multidisciplinary antibiotic stewardship at a hospital level. It also requires controlling pharmaceutical sales and the unregulated dispensing of antibiotic agents globally through policy initiatives to control emerging resistance through prevention.

show abstract

“…From X, we further derive two input matrices: 1) a matrix of masking vectors, m, that indicates whether the measurement is available (1) or missing (0); and 2) a δ matrix that records the time (in hours) since the last available fea-ture measurement. Examples of input matrix X, masking matrix m, and matrix δ are shown in Equations (1) - (4).…”

Section: Imputation Networkmentioning

confidence: 99%

“…Previous works on sepsis prediction have employed time varying Cox proportional hazards models [3], gradient boosting [4] and Gaussian process RNNs [5]. To tackle the challenges associated with this problem, we build a multitask neural architecture for early sepsis detection.…”

Section: Introductionmentioning

confidence: 99%

A Multi-Task Imputation and Classification Neural Architecture for Early Prediction of Sepsis from Multivariate Clinical Time Series

Chang¹,

Rubin²,

Boverman³

et al. 2019

2019 Computing in Cardiology Conference (CinC)

View full text Add to dashboard Cite

Early prediction of sepsis onset can notify clinicians to provide timely interventions to patients to improve their clinical outcomes. The key question motivating this work is: given a retrospective patient cohort consisting of multivariate clinical time series (e.g., vital signs and lab measurement) and patients' demographics, how to build a model to predict the onset of sepsis six hours earlier? To tackle this challenge, we first used a recurrent imputation for time series (RITS) approach to impute missing values in multivariate clinical time series. Second, we applied temporal convolutional networks (TCN) to the RITS-imputed data. Compared to other sequence prediction models, TCN can effectively control the size of sequence history. Third, when defining the loss function, we assigned custom timedependent weights to different types of errors. We achieved 9th place (team name = prna, utility score = 0.328) at the 2019 PhysioNet Computing in Cardiology Challenge, which evaluated our proposed model on a real-world sepsis patient cohort. At a follow-up 'hackathon' event, held by the challenge organizers, an improved version of our algorithm achieved 2nd place (utility score = 0.342).

show abstract

Evaluation of a machine learning algorithm for up to 48-hour advance prediction of sepsis using six vital signs

Cited by 114 publications

References 33 publications

Machine learning for the prediction of sepsis: a systematic review and meta-analysis of diagnostic test accuracy

Machine learning for the prediction of sepsis: a systematic review and meta-analysis of diagnostic test accuracy

Sepsis 2019: What Surgeons Need to Know

A Multi-Task Imputation and Classification Neural Architecture for Early Prediction of Sepsis from Multivariate Clinical Time Series

Contact Info

Product

Resources

About