Development of machine learning models to prognosticate chronic shunt-dependent hydrocephalus after aneurysmal subarachnoid hemorrhage

Abstract

Background Shunt-dependent hydrocephalus significantly complicates subarachnoid hemorrhage (SAH), and reliable prognosis methods have been sought in recent years to reduce morbidity and costs associated with delayed treatment or neglected onset. Machine learning (ML) defines modern data analysis techniques allowing accurate subject-based risk stratifications. We aimed at developing and testing different ML models to predict shunt-dependent hydrocephalus after aneurysmal SAH. Methods We consulted electronic records of patients with aneurysmal SAH treated at our institution between January 2013 and March 2019. We selected variables for the models according to the results of the previous works on this topic. We trained and tested four ML algorithms on three datasets: one containing binary variables, one considering variables associated with shunt-dependency after an explorative analysis, and one including all variables. For each model, we calculated AUROC, specificity, sensitivity, accuracy, PPV, and also, on the validation set, the NPV and the Matthews correlation coefficient (ϕ). Results Three hundred eighty-six patients were included. Fifty patients (12.9%) developed shunt-dependency after a mean follow-up of 19.7 (± 12.6) months. Complete information was retrieved for 32 variables, used to train the models. The best models were selected based on the performances on the validation set and were achieved with a distributed random forest model considering 21 variables, with a ϕ = 0.59, AUC = 0.88; sensitivity and specificity of 0.73 (C.I.: 0.39–0.94) and 0.92 (C.I.: 0.84–0.97), respectively; PPV = 0.59 (0.38–0.77); and NPV = 0.96 (0.90–0.98). Accuracy was 0.90 (0.82–0.95). Conclusions Machine learning prognostic models allow accurate predictions with a large number of variables and a more subject-oriented prognosis. We identified a single best distributed random forest model, with an excellent prognostic capacity (ϕ = 0.58), which could be especially helpful in identifying low-risk patients for shunt-dependency.

Outcome prediction in aneurysmal subarachnoid hemorrhage: a comparison of machine learning methods and established clinico-radiological scores

“…In neurosurgery, ML prediction models have been evaluated for a variety of pathologies with variable predictive performances (AUC 0.71 to 0.96) [26]. In the prediction of the occurrence of shunt-dependent hydrocephalus after aSAH, ML methods proved to be superior to traditional methods [24]. They included dynamic variables such as infections, treatment timing from symptom onset, and fever onset.…”

“…In our present study, we appliedamongst others-two of the most promising state-of-the-art ML techniques to predict functional outcome after aSAH: tree boosting and ANN. Both have shown considerable advances over traditional linear or logistic regression techniques in the past [12,38], even though traceability and comparability across different studies is reduced by substantial heterogeneity of clinical questions, input and output variables, and applied algorithms [19,24,26,30].…”

“…In this context, especially artificial neural nets (ANN) and methods of tree boosting, a decision treebased algorithm, showed better performance than traditional ML approaches such as linear and logistic regression for numerous applications [12,18,37]. However, the substantial heterogeneity of clinical questions, input and output variables, and applied algorithms may reduce traceability and reproducibility [24,32].…”

External Validation of a Neural Network Model in Aneurysmal Subarachnoid Hemorrhage: A Comparison With Conventional Logistic Regression Models

“…We have reviewed the literature to come up with a synopsis of the use of AI for outcome prediction in intracranial aneurysms (Table 4). In total, 24 studies were identified, and predicted outcomes encompassed occlusion rates after flow diversion, [26][27][28] perioperative outcomes after microsurgical treatment of unruptured aneurysms, 29 delayed cerebral ischemia, 9,[30][31][32][33][34][35] post-aSAH shunt-dependent hydrocephalus, 42,48 post-aSAH functional status at different time points (discharge until 1-year followup), 9,[35][36][37][38][39][40][41][42][43][44][46][47][48] and mortality 9,37,42,45 after aSAH. Most studies were published in the past 4 years (2018-2021), demonstrating that experience with this methodology is nascent and continuously evolving.…”

Machine Learning-Based Clinical Adjusted Selection of Predicting Risk Factors for Shunt Infection in Children

“…They concluded that building trust in these new technologies would require further improvement and more explanation of the ability and interpretability of applied models. Muscas and colleagues 9 analyzed different ML models to predict shunt-dependent hydrocephalus after aneurysmal subarachnoid hemorrhage. They suggested a single best distributed random forest model could be very helpful in identifying low-risk patients for shunt-dependency with an accuracy of 90%.…”