Validation of the PreOperative Score to predict Post-Operative Mortality (POSPOM) in Dutch non-cardiac surgery patients

Abstract

Background Standardized risk assessment tools can be used to identify patients at higher risk for postoperative complications and death. In this study, we validate the PreOperative Score to predict Post-Operative Mortality (POSPOM) for in-hospital mortality in a large cohort of non-cardiac surgery patients. In addition, the performance of POSPOM to predict postoperative complications was studied. Methods Data from the control cohort of the TRACE (routine posTsuRgical Anesthesia visit to improve patient outComE) study was analysed. POSPOM scores for each patient were calculated post-hoc. Observed in-hospital mortality was compared with predicted mortality according to POSPOM. Discrimination was assessed by receiver operating characteristic curves with C-statistics for in-hospital mortality and postoperative complications. To describe the performance of POSPOM sensitivity, specificity, negative predictive values, and positive predictive values were calculated. For in-hospital mortality, calibration was assessed by a calibration plot. Results In 2490 patients, the observed in-hospital mortality was 0.5%, compared to 1.3% as predicted by POSPOM. 27.1% of patients had at least one postoperative complication of which 22.4% had a major complication. For in-hospital mortality, POSPOM showed strong discrimination with a C-statistic of 0.86 (95% CI, 0.78–0.93). For the prediction of complications, the discrimination was poor to fair depending on the severity of the complication. The calibration plot showed poor calibration of POSPOM with an overestimation of in-hospital mortality. Conclusion Despite the strong discriminatory performance, POSPOM showed poor calibration with an overestimation of in-hospital mortality. Performance of POSPOM for the prediction of any postoperative complication was poor but improved according to severity.

Artificial Intelligence–enabled Decision Support in Surgery

“…The average AUROC for the best model across all 36 articles was 0.83; of the 8 articles with sample sizes of less than 2000 (ie, less than 1000 samples per class), 7 22,[26][27][28]31,35,37 (87.5%) had below-average (ie, less than 0.83) AUROC or accuracy. Twenty-four 9,18,20,[24][25][26][27][28][29][32][33][34][35]37,[39][40][41][43][44][45][46][47][48][49] articles (66.7%) presented confidence intervals for performance metrics. Among the 8 articles with sample sizes of less than 2000, 5 [26][27][28]35,37 presented confidence intervals.…”

“…Twentythree articles 9,19,21,[23][24][25]27,29,30,[32][33][34]36,37,39,41,42,[44][45][46][48][49][50] (63.9%) reported precision metrics (area under the precision-recall curve, positive predictive value, or F1 score). Twenty-five articles 9,20,21,[23][24][25][26][27][28]31,33,34,36,38,40,[42][43][44][45][46][47][48][49][50] (69.4%) included explainability mechanisms to convey the relative importance of input features in determining outputs. Thirteen articles 9,16,17,…”

“…Twenty-five articles 9,20,21,[23][24][25][26][27][28]31,33,34,36,38,40,[42][43][44][45][46][47][48][49][50] (69.4%) included explainability mechanisms to convey the relative importance of input features in determining outputs. Thirteen articles 9,16,17,20,25,27,29,30,35,38,45,46,50 (36.1%) presented a framework that could be used for clinical implementation; none of the articles assessed the efficacy of clinical implementation. in 5 articles performed external or real-time validation; 1 in 5 likely might have had overfitting and lower predictive performance due to small sample sizes; only two-thirds presented confidence intervals for performance metrics; less than two-thirds reported measures of precision; and none of the articles reported analyses across sex or racial groups or assessed the efficacy of clinical implementation.…”

“…None of the articles described equity analyses in which model performance was stratified by sex or race. Twenty-three articles9,19,21,23–25,27,29,30,32–34,36,37,39,41,42,44–46,48–50 (63.9%) reported precision metrics (area under the precision-recall curve, positive predictive value, or F1 score). Twenty-five articles9,20,21,23–28,31,33,34,36,38,40,42–50 (69.4%) included explainability mechanisms to convey the relative importance of input features in determining outputs.…”

National multicenter prospective observational study “The role of concomitant diseases in poSTOPerative complications RISK stratification — STOPRISK”: study protocol

“…С другой стороны, в масштабных обсервационных исследованиях последних лет, проведенных в ряде за рубежных стран, сопутствующие заболевания не были определены как неза висимые предикторы развития по слеоперационных осложнений [5], а системы предопе рационной оценки, основанные на сопутствующих за болеваниях, такие как POSPOM -Preoperative Score to Predict Postoperative Mortality, демонстрируют проти воречивую прогностическую ценность в нека рдиальной хирургии от недо оценки риска летального исхода [20] до ее переоценки [21].…”