Development and Validation of a Deep Neural Network Model for Prediction of Postoperative In-hospital Mortality

Abstract

Editor’s Perspective What We Already Know about This Topic What This Article Tells Us That Is New Background The authors tested the hypothesis that deep neural networks trained on intraoperative features can predict postoperative in-hospital mortality. Methods The data used to train and validate the algorithm consists of 59,985 patients with 87 features extracted at the end of surgery. Feed-forward networks with a logistic output were trained using stochastic gradient descent with momentum. The deep neural networks were trained on 80% of the data, with 20% reserved for testing. The authors assessed improvement of the deep neural network by adding American Society of Anesthesiologists (ASA) Physical Status Classification and robustness of the deep neural network to a reduced feature set. The networks were then compared to ASA Physical Status, logistic regression, and other published clinical scores including the Surgical Apgar, Preoperative Score to Predict Postoperative Mortality, Risk Quantification Index, and the Risk Stratification Index. Results In-hospital mortality in the training and test sets were 0.81% and 0.73%. The deep neural network with a reduced feature set and ASA Physical Status classification had the highest area under the receiver operating characteristics curve, 0.91 (95% CI, 0.88 to 0.93). The highest logistic regression area under the curve was found with a reduced feature set and ASA Physical Status (0.90, 95% CI, 0.87 to 0.93). The Risk Stratification Index had the highest area under the receiver operating characteristics curve, at 0.97 (95% CI, 0.94 to 0.99). Conclusions Deep neural networks can predict in-hospital mortality based on automatically extractable intraoperative data, but are not (yet) superior to existing methods.

Prediction of mortality following pediatric heart transplant using machine learning algorithms

“…In line with prior studies, variables with >10% missing values were excluded from consideration, 17,20 and data were split randomly as 80% training and 20% testing (stratified on patient mortality at last known follow-up, to maintain approximately even mortality rates between training and testing sets). 21 Using the training data, categorical measures were divided into binary variables to retain clinically relevant distinctions between categories, while eliminating potentially inaccurate ordinal relationships between categories. For highly collinear variable pairs, the variable having the largest mean absolute correlation with other variables in the training set was excluded from the set.…”

Preoperative predictions of in-hospital mortality using electronic medical record data

“…To determine which features were most important to the classification models, we 144 examined the model weights for linear models, the feature (Gini) importance for the 145 random forest models, and the feature weight (number of times a feature appears in a 146 tree) for the gradient boosted tree models. Previous work [11] has shown that integrating a measure of preoperative risk into a 149 postoperative mortality risk prediction model increases the model performance. We Lee et al [11], we replaced the ASA status feature with the preoperative risk score 154 (generated using the random forest model with preoperative features and imputed-ASA 155 scores) and trained the model on the same cohort used for preoperative risk prediction.…”

“…Previous work [11] has shown that integrating a measure of preoperative risk into a 149 postoperative mortality risk prediction model increases the model performance. We Lee et al [11], we replaced the ASA status feature with the preoperative risk score 154 (generated using the random forest model with preoperative features and imputed-ASA 155 scores) and trained the model on the same cohort used for preoperative risk prediction. 156 The intraoperative data were preprocessed in the same manner as described in [11].…”

“…We compare the performance of 27 our model to existing clinical risk scores (ASA Score and Charlson Comorbidity 28 Score [6]) and show that our model has a considerably higher area under the curve when 29 compared to these. Last, we integrate our model with a previously published model [11] 30 to automatically predict in-hospital mortality immediately after surgery in order to 31 quantify the change in risk during the perioperative period, and we show that, while the 32 majority of patients do not exhibit a substantial change in risk, a subset of the 33 population does show a large change in risk and, therefore, the risk should be monitored 34 at multiple time points. 35 1 Materials and methods 36 1.1 Data source and extraction 37 All data for this study were extracted from the Perioperative Data Warehouse (PDW), 38 a custom built, robust data warehouse containing all patients who have undergone 39 surgery at UCLA Health since the implementation of our EMR (EPIC Systems, 40 Madison, WI) in March 2013.…”

“…We Lee et al [11], we replaced the ASA status feature with the preoperative risk score 154 (generated using the random forest model with preoperative features and imputed-ASA 155 scores) and trained the model on the same cohort used for preoperative risk prediction. 156 The intraoperative data were preprocessed in the same manner as described in [11]. We 157 then compared the area under the ROC of the postoperative model trained using the 158 ASA status and intraoperative features to the postoperative model that was trained 159 using the preoperative risk score and intraoperative features.…”