Predicting Mortality in Intensive Care Unit Patients With Heart Failure Using an Interpretable Machine Learning Model: Retrospective Cohort Study

Abstract

Background Heart failure (HF) is a common disease and a major public health problem. HF mortality prediction is critical for developing individualized prevention and treatment plans. However, due to their lack of interpretability, most HF mortality prediction models have not yet reached clinical practice. Objective We aimed to develop an interpretable model to predict the mortality risk for patients with HF in intensive care units (ICUs) and used the SHapley Additive exPlanation (SHAP) method to explain the extreme gradient boosting (XGBoost) model and explore prognostic factors for HF. Methods In this retrospective cohort study, we achieved model development and performance comparison on the eICU Collaborative Research Database (eICU-CRD). We extracted data during the first 24 hours of each ICU admission, and the data set was randomly divided, with 70% used for model training and 30% used for model validation. The prediction performance of the XGBoost model was compared with three other machine learning models by the area under the curve. We used the SHAP method to explain the XGBoost model. Results A total of 2798 eligible patients with HF were included in the final cohort for this study. The observed in-hospital mortality of patients with HF was 9.97%. Comparatively, the XGBoost model had the highest predictive performance among four models with an area under the curve (AUC) of 0.824 (95% CI 0.7766-0.8708), whereas support vector machine had the poorest generalization ability (AUC=0.701, 95% CI 0.6433-0.7582). The decision curve showed that the net benefit of the XGBoost model surpassed those of other machine learning models at 10%~28% threshold probabilities. The SHAP method reveals the top 20 predictors of HF according to the importance ranking, and the average of the blood urea nitrogen was recognized as the most important predictor variable. Conclusions The interpretable predictive model helps physicians more accurately predict the mortality risk in ICU patients with HF, and therefore, provides better treatment plans and optimal resource allocation for their patients. In addition, the interpretable framework can increase the transparency of the model and facilitate understanding the reliability of the predictive model for the physicians.

Machine learning-based in-hospital mortality risk prediction tool for intensive care unit patients with heart failure

“…According to previous researches (13,14), this study documented demographic characteristics (age, sex, weight and height), severity score [Sequential Organ Failure Assessment (SOFA) and Acute Physiology Score III (APS III)], comorbidities [diabetes, hypertension, severe liver disease, chronic obstructive pulmonary disease (COPD), and myocardial infarction], medication information [dopamine, norepinephrine, dobutamine, epinephrine, phenylephrine, vasopressin, milrinone, furosemide, beta blocker and angiotensin converting enzyme I (AECI)/ Angiotensin II receptor blocker (ARB)], mechanical ventilation, continuous renal replacement therapy (CRRT), vital signs at admission (heart rate, respiratory rate, SpO 2 , systolic blood pressure and mean blood pressure), and laboratory tests [white blood cell count (WBC), basophils, eosinophils, lymphocytes, monocytes, neutrophils, red blood cells count (RBC), hematocrit, hemoglobin, mean corpuscular volume (MCV), mean corpuscular hemoglobin content (MCHC), mean hemoglobin concentration (MHC), red blood cell distribution width (RDW), platelets, albumin, aspartate aminotransferase (AST) alanine aminotransferase (ALT), alkaline phosphatase (ALP), total bilirubin, urea nitrogen (BUN), creatinine, glucose, sodium, calcium, chloride, potassium, anion gap, bicarbonate, international normalized ratio (INR), thromboplastin time (PTT), prothrombin time (PT)] at admission. The same features in eICU-CRD were extracted to achieve external validation of those models.…”

Development and validation of a machine learning-based predictive model for secondary post-tonsillectomy hemorrhage

“…Clinical net benefit is defined as the minimum probability of disease, when further intervention would be warranted. The plot measures the net benefit at different threshold probabilities ( 12 ). The blue line in Figure 3 indicates the hypothesis that all patients received the intervention, while the black line indicates that no patients received the intervention due to the heterogeneity of the study population.…”

“…In addition, decision curve analysis (DCA) was performed to assess the utility of the decision model by quantifying the net benefit under different threshold probabilities. The interpretation of the prediction model was performed by SHAP, a unified approach that precisely calculates the contribution and impact of each variable on the final prediction ( 12 ). Each observation in the dataset can be interpreted by a specific SHAP value.…”

“…Machine learning models and advanced statistical tools can be used to predict the outcome of complex datasets based on iterative learning, thereby making these models more accurate and stable through the selection of variable features. Therefore, more researchers advocate the use of new predictive models based on machine learning to support personalized treatment for patients (11)(12)(13). To the best of our knowledge, there is no predictive model for PTH based on machine learning.…”

A Hybrid Modeling Framework for Generalizable and Interpretable Predictions of ICU Mortality: Leveraging ICD Codes in a Multi-Hospital Study of Mechanically Ventilated Influenza Patients

“…A notable example is the Sequential Organ Failure Assessment (SOFA) score; while initially designed to assess organ dysfunction in sepsis patients 7 , it was later noticed that repeated measurements correlated with mortality 8 , it even became essential in the definition of sepsis in the frame of the Sepsis-3 consensus 26 . Other examples of interpretable clinical predictive models and scoring systems in medicine are, to name a few, for mortality prediction in patients with heart failure 27 , or mortality in general 28 .…”