Assessing the utility of deep neural networks in predicting postoperative surgical complications: a retrospective study

Abstract

Background Early detection of postoperative complications, including organ failure, is pivotal in the initiation of targeted treatment strategies aimed at attenuating organ damage. In an era of increasing health-care costs and limited financial resources, identifying surgical patients at a high risk of postoperative complications and providing personalised precision medicine-based treatment strategies provides an obvious pathway for reducing patient morbidity and mortality. We aimed to leverage deep learning to create, through training on structured electronic health-care data, a multilabel deep neural network to predict surgical postoperative complications that would outperform available models in surgical risk prediction.Methods In this retrospective study, we used data on 58 input features, including demographics, laboratory values, and 30-day postoperative complications, from the American College of Surgeons (ACS) National Surgical Quality Improvement Program database, which collects data from 722 hospitals from around 15 countries. We queried the entire adult (≥18 years) database for patients who had surgery between Jan 1, 2012, and Dec 31, 2018. We then identified all patients who were treated at a large midwestern US academic medical centre, excluded them from the base dataset, and reserved this independent group for final model testing. We then randomly created a training set and a validation set from the remaining cases. We developed three deep neural network models with increasing numbers of input variables and so increasing levels of complexity. Output variables comprised mortality and 18 different postoperative complications. Overall morbidity was defined as any of 16 postoperative complications. Model performance was evaluated on the test set using the area under the receiver operating characteristic curve (AUC) and compared with previous metrics from the ACS-Surgical Risk Calculator (ACS-SRC). We evaluated resistance to changes in the underlying patient population on a subset of the test set, comprising only patients who had emergency surgery. Results were also compared with the Predictive OpTimal Trees in Emergency Surgery Risk (POTTER) calculator. Findings 5 881 881 surgical patients, with 2941 unique Current Procedural Terminology codes, were included in this study, with 4 694 488 in the training set, 1 173 622 in the validation set, and 13 771 in the test set. The mean AUCs for the validation set were 0•864 (SD 0•053) for model 1, 0•871 (0•055) for model 2, and 0•882 (0•053) for model 3. The mean AUCs for the test set were 0•859 (SD 0•063) for model 1, 0•863 (0•064) for model 2, and 0•874 (0•061) for model 3. The mean AUCs of each model outperformed previously published performance metrics from the ACS-SRC, with a direct correlation between increasing model complexity and performance. Additionally, when tested on a subgroup of patients who had emergency surgery, our models outperformed previously published POTTER metrics. Interpretation We have developed unified prediction models, based on deep ...

Development and Validation of a Personalized Model With Transfer Learning for Acute Kidney Injury Risk Estimation Using Electronic Health Records

“…Global models can capture knowledge that is generalizable to a population but may ignore information that is specific to an individual or a subpopulation. 13 , 14 , 15 , 16 An alternative is subgroup modeling, 17 , 18 , 19 , 20 , 21 , 22 , 23 which is stratifying a population into subgroups according to patient differences and then building models for each subgroup. These subgroups, however, are defined by preexisting knowledge.…”

“…16 subgroups, the subgroup model in 11 subgroups, and the subgroup model with transfer learning in 9 subgroups. For example, among patients older than 65 years, AUROC was 0.76 (95% CI, 0.74-0.77) for the personalized model with transfer learning, 0.73 (95% CI, 0.72-0.75; P < .001) for the global model, 0.71 (95% CI, 0.70-0.72; P < .001) for the subgroup model, and 0.73 (95% CI, 0.72-0.75; with transfer learning, whereas the reported AUROC of the previous model built with a sample size that was 280 times greater than the present sample size was 0.70 (95% CI, 0.69-0.71; P = .04).…”

A multicenter prospective study on postoperative pulmonary complications prediction in geriatric patients with deep neural network model

“…Deep learning has the advantage of learning directly from natural language data without the need for manual processing ( 32 , 33 ). In our study, natural language data contained descriptions about principal diagnoses and comorbidities.…”

“…To process natural language data, we performed embedding using MedBERT and mean-pooling, instead of traditional one-hot encoding. With one-hot encoding, natural language data are actually transformed into binary variables according to the presence or absence of particular words ( 26 ), which may hinder the learning of potential relationships between descriptions ( 32 ). Embedding does not regard two principal diagnoses as completely different categories.…”

Application of Machine Learning Models to Predict Recurrence After Surgical Resection of Nonmetastatic Renal Cell Carcinoma

“…ML has the ability to discern associations between variables that are not obvious or detected by conventional tools [9]. ML has become more and more popular with a wide range of applications that seem promising in various medical fields, including medical imaging [10], detection of diabetic retinopathy [11], or prediction of surgical complications [12].…”