Prediction of Postoperative Deterioration in Cardiac Surgery Patients Using Electronic Health Record and Physiologic Waveform Data

Abstract

Background Postoperative hemodynamic deterioration among cardiac surgical patients can indicate or lead to adverse outcomes. Whereas prediction models for such events using electronic health records or physiologic waveform data are previously described, their combined value remains incompletely defined. The authors hypothesized that models incorporating electronic health record and processed waveform signal data (electrocardiogram lead II, pulse plethysmography, arterial catheter tracing) would yield improved performance versus either modality alone. Methods Intensive care unit data were reviewed following elective adult cardiac surgical procedures at an academic center between 2013-2020. Model features included electronic health record features and physiologic waveforms. Tensor decomposition was used for waveform feature reduction. Machine learning-based prediction models included a 2013-2017 training set and a 2017-2020 temporal holdout test set. The primary outcome was a postoperative deterioration event, defined as a composite of low cardiac index <2.0mL/min/m2, mean arterial pressure <55mmHg sustained ≥120 minutes, new or escalated inotrope/vasopressor infusion, epinephrine bolus ≥1mg, or intensive care unit mortality. Prediction models analyzed data 8 hours prior to events. Results Among 1,555 cases, 185 (12%) experienced 276 deterioration events, most commonly including low cardiac index (7.0% of patients), new inotrope (1.9%), and sustained hypotension (1.4%). The best performing model on the 2013-2017 training set yielded a c-statistic of 0.803 (95% CI 0.799-0.807) although performance was substantially lower in the 2017-2020 test set (0.709, 0.705-0.712). Test set performance of the combined model was greater than corresponding models limited to solely electronic health record features (0.641, 0.637-0.646) or waveform features (0.697, 0.693-0.701). Conclusions Clinical deterioration prediction models combining electronic health record data and waveform data were superior to either modality alone, and performance of combined models was primarily driven by waveform data. Decreased performance of prediction models during temporal validation may be explained by dataset shift, a core challenge of healthcare prediction modeling.

Artificial Intelligence–enabled Decision Support in Surgery

“…The average AUROC for the best model across all 36 articles was 0.83; of the 8 articles with sample sizes of less than 2000 (ie, less than 1000 samples per class), 7 22,[26][27][28]31,35,37 (87.5%) had below-average (ie, less than 0.83) AUROC or accuracy. Twenty-four 9,18,20,[24][25][26][27][28][29][32][33][34][35]37,[39][40][41][43][44][45][46][47][48][49] articles (66.7%) presented confidence intervals for performance metrics. Among the 8 articles with sample sizes of less than 2000, 5 [26][27][28]35,37 presented confidence intervals.…”

“…Twenty-four 9,18,20,[24][25][26][27][28][29][32][33][34][35]37,[39][40][41][43][44][45][46][47][48][49] articles (66.7%) presented confidence intervals for performance metrics. Among the 8 articles with sample sizes of less than 2000, 5 [26][27][28]35,37 presented confidence intervals. Among these 5 articles, 4 [26][27][28]35 had wide confidence intervals ranging from 0.15 to 0.35, and 1, 37 which used waveform input features, had a narrow confidence interval of 0.01 (albeit with moderate performance in predicting postoperative deterioration, AUROC 0.71).…”

“…Among the 8 articles with sample sizes of less than 2000, 526–28,35,37 presented confidence intervals. Among these 5 articles, 426–28,35 had wide confidence intervals ranging from 0.15 to 0.35, and 1,37 which used waveform input features, had a narrow confidence interval of 0.01 (albeit with moderate performance in predicting postoperative deterioration, AUROC 0.71). Among all 36 articles, 531,33,35,39,40 (13.8%) performed external validation, 29,29 (5.6%) performed real-time validation, and 29 (80.6%) performed internal validation only.…”

“…Table 1 summarizes study populations, model architecture, performance, validation type, and explainability, and whether precision and implementation frameworks were reported. The overall sample size range was 163-2,882,526, with 8 22,23,[26][27][28]31,35,37 of 36 articles (22.2%) featuring sample sizes of less than 2000. The average AUROC for the best model across all 36 articles was 0.83; of the 8 articles with sample sizes of less than 2000 (ie, less than 1000 samples per class), 7 22,[26][27][28]31,35,37 (87.5%) had below-average (ie, less than 0.83) AUROC or accuracy.…”

“…The overall sample size range was 163-2,882,526, with 8 22,23,[26][27][28]31,35,37 of 36 articles (22.2%) featuring sample sizes of less than 2000. The average AUROC for the best model across all 36 articles was 0.83; of the 8 articles with sample sizes of less than 2000 (ie, less than 1000 samples per class), 7 22,[26][27][28]31,35,37 (87.5%) had below-average (ie, less than 0.83) AUROC or accuracy. Twenty-four 9,18,20,[24][25][26][27][28][29][32][33][34][35]37,[39][40][41][43][44][45][46][47][48][49] articles (66.7%) presented confidence intervals for performance metrics.…”