Postoperative delirium prediction using machine learning models and preoperative electronic health record data

Abstract

Background Accurate, pragmatic risk stratification for postoperative delirium (POD) is necessary to target preventative resources toward high-risk patients. Machine learning (ML) offers a novel approach to leveraging electronic health record (EHR) data for POD prediction. We sought to develop and internally validate a ML-derived POD risk prediction model using preoperative risk features, and to compare its performance to models developed with traditional logistic regression. Methods This was a retrospective analysis of preoperative EHR data from 24,885 adults undergoing a procedure requiring anesthesia care, recovering in the main post-anesthesia care unit, and staying in the hospital at least overnight between December 2016 and December 2019 at either of two hospitals in a tertiary care health system. One hundred fifteen preoperative risk features including demographics, comorbidities, nursing assessments, surgery type, and other preoperative EHR data were used to predict postoperative delirium (POD), defined as any instance of Nursing Delirium Screening Scale ≥2 or positive Confusion Assessment Method for the Intensive Care Unit within the first 7 postoperative days. Two ML models (Neural Network and XGBoost), two traditional logistic regression models (“clinician-guided” and “ML hybrid”), and a previously described delirium risk stratification tool (AWOL-S) were evaluated using the area under the receiver operating characteristic curve (AUC-ROC), sensitivity, specificity, positive likelihood ratio, and positive predictive value. Model calibration was assessed with a calibration curve. Patients with no POD assessments charted or at least 20% of input variables missing were excluded. Results POD incidence was 5.3%. The AUC-ROC for Neural Net was 0.841 [95% CI 0. 816–0.863] and for XGBoost was 0.851 [95% CI 0.827–0.874], which was significantly better than the clinician-guided (AUC-ROC 0.763 [0.734–0.793], p < 0.001) and ML hybrid (AUC-ROC 0.824 [0.800–0.849], p < 0.001) regression models and AWOL-S (AUC-ROC 0.762 [95% CI 0.713–0.812], p < 0.001). Neural Net, XGBoost, and ML hybrid models demonstrated excellent calibration, while calibration of the clinician-guided and AWOL-S models was moderate; they tended to overestimate delirium risk in those already at highest risk. Conclusion Using pragmatically collected EHR data, two ML models predicted POD in a broad perioperative population with high discrimination. Optimal application of the models would provide automated, real-time delirium risk stratification to improve perioperative management of surgical patients at risk for POD.

Postoperative delirium in geriatric patients with hip fractures

“…According to its procedure-specific delirium risk score, hip fracture surgery belongs to the high-risk level, which needs continuing delirium prevention interventions after discharge from anesthesia care. Bishara et al (2022) developed a delirium risk prediction model based on machine learning, which demonstrated excellent calibration compared with models developed with traditional logistic regression. It is not limited by time and space and can be repeated and standardized to assess patients.…”

Machine-learning model predicting postoperative delirium in older patients using intraoperative frontal electroencephalographic signatures

“…Even when including little prior knowledge about the patient, the cross-validation results for Propofol and Sevoflurane are comparable to the POD prediction results achieved by Bishara et al (2022), using electronic health record data of 24,885 adults, reporting an average AUC-ROC of 0.82 for older patients. This shows the potential of incorporating Limiting the patient data to readily available information, without any additional examinations or measurements, is more practical.…”

“…In addition to these classical risk factors, several machine learning approaches have been developed for predicting the risk of POD with the goal of early diagnosis and prevention during or directly after surgery. These approaches rely on different databases, such as electronic health record data (Wang et al, 2020;Bishara et al, 2022) or MRI data (Kyeong et al, 2018) and often focus on one type of surgery (Kyeong et al, 2018;Wang et al, 2020). EEG data has been used as well (van Sleuwen et al, 2022;Tesh et al, 2022), however, not to predict POD specifically, but to predict clinical outcomes and the severity of delirium in general for diagnosis and treatment once patients might have developed delirium on the ward.…”

Perioperative Brain Health in the Older Adult: A Patient Safety Imperative

“…POD prediction models that use ML and AI technologies have been published recently. [89][90][91] One such model 91 used 115 predictive features from electronic health record data with encouraging results, as defined by the area under the receiver operating characteristic curve.…”