Predicting Outcomes Following Endovascular Abdominal Aortic Aneurysm Repair Using Machine Learning

Abstract

Objective: To develop machine learning (ML) models that predict outcomes following endovascular aneurysm repair (EVAR) for abdominal aortic aneurysm (AAA). Summary Background Data: EVAR carries non-negligible peri-operative risks; however, there are no widely used outcome prediction tools. Methods: The National Surgical Quality Improvement Program targeted database was used to identify patients who underwent EVAR for infrarenal AAA between 2011-2021. Input features included 36 pre-operative variables. The primary outcome was 30-day major adverse cardiovascular event (MACE; composite of myocardial infarction, stroke, or death). Data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, 6 ML models were trained using pre-operative features. The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). Model robustness was evaluated with calibration plot and Brier score. Subgroup analysis was performed to assess model performance based on age, sex, race, ethnicity, and prior AAA repair. Results: Overall, 16,282 patients were included. The primary outcome of 30-day MACE occurred in 390 (2.4%) patients. Our best performing prediction model was XGBoost, achieving an AUROC (95% CI) of 0.95 (0.94-0.96) compared to logistic regression [0.72 (0.70 – 0.74)]. The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.06. Model performance remained robust on all subgroup analyses. Conclusions: Our newer ML models accurately predict 30-day outcomes following EVAR using pre-operative data and perform better than logistic regression. Our automated algorithms can guide risk mitigation strategies for patients being considered for EVAR.

Predicting Outcomes Following Lower Extremity Endovascular Revascularization Using Machine Learning

“…Our work has demonstrated that ML algorithms can perform better than logistic regression and existing tools for predicting outcomes following vascular interventions. 19 , 20 , 21 , 22 , 23 In particular, we showed that procedure‐specific ML models can achieve better performance than ML algorithms developed on heterogenous surgical populations, such as the NSQIP‐based ML model by Bonde and colleagues that was trained on a pooled data set of over 2900 different surgical procedures. 19 , 20 , 21 , 22 , 23 , 24 Our vascular procedure‐specific models achieved area under the receiver operating characteristic curve (AUROC) values ≥0.90 whereas the algorithm by Bonde et al attained AUROCs between 0.85 and 0.88.…”

“… 19 , 20 , 21 , 22 , 23 In particular, we showed that procedure‐specific ML models can achieve better performance than ML algorithms developed on heterogenous surgical populations, such as the NSQIP‐based ML model by Bonde and colleagues that was trained on a pooled data set of over 2900 different surgical procedures. 19 , 20 , 21 , 22 , 23 , 24 Our vascular procedure‐specific models achieved area under the receiver operating characteristic curve (AUROC) values ≥0.90 whereas the algorithm by Bonde et al attained AUROCs between 0.85 and 0.88. 19 , 20 , 21 , 22 , 23 , 24 Therefore, there may be value in building a ML model specifically designed to predict outcomes following lower extremity endovascular revascularization.…”

“… 18 Previously, we developed ML algorithms to predict outcomes following carotid, aortic, and open lower extremity interventions. 19 , 20 , 21 , 22 , 23 However, there exists no ML algorithm specifically designed to predict outcomes following endovascular revascularization for PAD. Our work has demonstrated that ML algorithms can perform better than logistic regression and existing tools for predicting outcomes following vascular interventions.…”

Letter re: Machine Learning to Predict Outcomes Following Endovascular Aortic Aneurysm Repair

“…Using the National Surgical Quality Improvement Program targeted database, a recent study trained 6 ML models using a cohort of 16 282 patients to predict 30-day major adverse cardiovascular events after EVAR. 5 The best performing prediction model (XGBoost) achieved an area under the curve of .95 (.94-.96) and performed better compared to logistic regression [.72 (.70 - .74)]. 5 Altogether, ML-models offer new perspectives to predict complications after EVAR such as mortality, endoleak, risk of re-intervention, adverse EVAR-related events or major adverse cardiovascular events.…”

“…5 The best performing prediction model (XGBoost) achieved an area under the curve of .95 (.94-.96) and performed better compared to logistic regression [.72 (.70 - .74)]. 5 Altogether, ML-models offer new perspectives to predict complications after EVAR such as mortality, endoleak, risk of re-intervention, adverse EVAR-related events or major adverse cardiovascular events. Such approach might help in the future to better assess the prognosis of patients, plan the intervention and optimize the follow-up of patients.…”