Augmenting mortality prediction with medication data and machine learning models

Abstract

Background: In critically ill patients, complex relationships exist among patient disease factors, medication management, and mortality. Considering the potential for nonlinear relationships and the high dimensionality of medication data, machine learning and advanced regression methods may offer advantages over traditional regression techniques. The purpose of this study was to evaluate the role of different modeling approaches incorporating medication data for mortality prediction. Methods: This was a single-center, observational cohort study of critically ill adults. A random sample of 991 adults admitted ≥ 24 hours to the intensive care unit (ICU) from 10/2015 to 10/2020 were included. Models to predict hospital mortality at discharge were created. Models were externally validated against a temporally separate dataset of 4,878 patients. Potential mortality predictor variables (n=27, together with 14 indicators for missingness) were collected at baseline (age, sex, service, diagnosis) and 24 hours (illness severity, supportive care use, fluid balance, laboratory values, MRC-ICU score, and vasopressor use) and included in all models. The optimal traditional (equipped with linear predictors) logistic regression model and optimal advanced (equipped with nature splines, smoothing splines, and local linearity) logistic regression models were created using stepwise selection by Bayesian information criterion (BIC). Supervised, classification-based ML models [e.g., Random Forest, Support Vector Machine (SVM), and XGBoost] were developed. Area under the receiver operating characteristic (AUROC), positive predictive value (PPV), and negative predictive value (NPV) were compared among different mortality prediction models. Results: A model including MRC-ICU in addition to SOFA and APACHE II demonstrated an AUROC of 0.83 for hospital mortality prediction, compared to AUROCs of 0.72 and 0.81 for APACHE II and SOFA alone. Machine learning models based on Random Forest, SVM, and XGBoost demonstrated AUROCs of 0.83, 0.85, and 0.82, respectively. Accuracy of traditional regression models was similar to that of machine learning models. MRC-ICU demonstrated a moderate level of feature importance in both XGBoost and Random Forest. Across all ten models, performance was lower on the validation set. Conclusions: While medication data were not included as a significant predictor in regression models, addition of MRC-ICU to severity of illness scores (APACHE II and SOFA) improved AUROC for mortality prediction. Machine learning methods did not improve model performance relative to traditional regression methods.