Robust Prediction of Non-home Discharge After Thoracolumbar Spine Surgery With Ensemble Machine Learning and Validation on a Nationwide Cohort

Valliani, Aly; Kim, Nora C.; Martini, Michael L; Gal, Jonathan S.; Neifert, Sean N; Feng, Rui; Geng, Eric; Kim, Jun; Cho, Samuel K.; Oermann, Eric K.; Caridi, John M.

doi:10.1016/j.wneu.2022.05.105

Cited by 5 publications

(2 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The C-statistic of 0.87 suggests most NHD can be anticipated preoperatively by considering RAI frailty score, surgical indication, the timing of surgery (elective vs. nonelective), BMI, and several A model with this level of diagnostic accuracy is superior compared to similar models in previous literature. 36,37 The predictive model bears clinically translatable knowledge that may be used to reduce poor outcomes among spine surgery patients with augmentation of surgical decision-making or perioperative care.…”

Section: Discussionmentioning

confidence: 99%

Endoscopic and Nonendoscopic Approaches to Single-Level Lumbar Spine Decompression: Propensity Score-Matched Comparative Analysis and Frailty-Driven Predictive Model

et al. 2023

View full text Add to dashboard Cite

Objective: The endoscopic spine surgery (ESS) approach is associated with high levels of patient satisfaction, shorter recovery time, and reduced complications. The present study reports multicenter, international data, comparing ESS and non-ESS approaches for singlelevel lumbar decompression, and proposes a frailty-driven predictive model for nonhome discharge (NHD) disposition.Methods: Cases of ESS and non-ESS lumbar spine decompression were queried from the American College of Surgeons National Surgical Quality Improvement Program database (2017–2020). Propensity score matching was performed on baseline characteristics frailty score (measured by risk analysis index [RAI] and modified frailty index-5 [mFI-5]). The primary outcome of interest was NHD disposition. A predictive model was built using logistic regression with RAI as the primary driver.Results: Single-level nonfusion spine lumbar decompression surgery was performed in 38,686 patients. Frailty, as measured by RAI, was a reliable predictor of NHD with excellent discriminatory accuracy in receiver operating characteristic (ROC) curve analysis: C-statistic: 0.80 (95% confidence interval [CI], 0.65–0.94) in ESS cohort, C-statistic: 0.75 (95% CI, 0.73–0.76) overall cohort. After propensity score matching, there was a reduction in total operative time (89 minutes vs. 103 minutes, p = 0.049) and hospital length of stay (LOS) (0.82 days vs. 1.37 days, p < 0.001) in patients treated endoscopically. In ROC curve analysis, the frailty-driven predictive model performed with excellent diagnostic accuracy for the primary outcome of NHD (C-statistic: 0.87; 95% CI, 0.85–0.88).Conclusion: After frailty-based propensity matching, ESS is associated with reduced operative time, shorter hospital LOS, and decreased NHD. The RAI frailty-driven model predicts NHD with excellent diagnostic accuracy and may be applied to preoperative decisionmaking with a user-friendly calculator: nsgyfrailtyoutcomeslab.shinyapps.io/lumbar_decompression_dischargedispo.

show abstract

Section: Discussionmentioning

confidence: 99%

Endoscopic and Nonendoscopic Approaches to Single-Level Lumbar Spine Decompression: Propensity Score-Matched Comparative Analysis and Frailty-Driven Predictive Model

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Valliani et al predicted non-home discharge with remarkable AUROCs, PPV, and NPV. The study also presented a well-calibrated model through a calibration plot, although the plot did not display true probability and predicted risks greater than 0.8 [18]. Despite these models' solid performance on the metrics reported, studies in this category failed to report other metrics crucial for model evaluation.…”

Section: Characteristics Of the Included Studiesmentioning

confidence: 98%

Limitations in Evaluating Machine Learning Models for Imbalanced Binary Outcome Classification in Spine Surgery: A Systematic Review

Ghanem,

Ghaith,

El-Hajj

et al. 2023

Brain Sciences

View full text Add to dashboard Cite

Clinical prediction models for spine surgery applications are on the rise, with an increasing reliance on machine learning (ML) and deep learning (DL). Many of the predicted outcomes are uncommon; therefore, to ensure the models’ effectiveness in clinical practice it is crucial to properly evaluate them. This systematic review aims to identify and evaluate current research-based ML and DL models applied for spine surgery, specifically those predicting binary outcomes with a focus on their evaluation metrics. Overall, 60 papers were included, and the findings were reported according to the PRISMA guidelines. A total of 13 papers focused on lengths of stay (LOS), 12 on readmissions, 12 on non-home discharge, 6 on mortality, and 5 on reoperations. The target outcomes exhibited data imbalances ranging from 0.44% to 42.4%. A total of 59 papers reported the model’s area under the receiver operating characteristic (AUROC), 28 mentioned accuracies, 33 provided sensitivity, 29 discussed specificity, 28 addressed positive predictive value (PPV), 24 included the negative predictive value (NPV), 25 indicated the Brier score with 10 providing a null model Brier, and 8 detailed the F1 score. Additionally, data visualization varied among the included papers. This review discusses the use of appropriate evaluation schemes in ML and identifies several common errors and potential bias sources in the literature. Embracing these recommendations as the field advances may facilitate the integration of reliable and effective ML models in clinical settings.

show abstract

Explainable Machine Learning Approach to Prediction of Prolonged Intesive Care Unit Stay in Adult Spinal Deformity Patients: Machine Learning Outperforms Logistic Regression

Zaidat,

Kurapatti,

Gal

et al. 2024

Global Spine Journal

View full text Add to dashboard Cite

Study Design Retrospective cohort study. Objectives Prolonged ICU stay is a driver of higher costs and inferior outcomes in Adult Spinal Deformity (ASD) patients. Machine learning (ML) models have recently been seen as a viable method of predicting pre-operative risk but are often ‘black boxes’ that do not fully explain the decision-making process. This study aims to demonstrate ML can achieve similar or greater predictive power as traditional statistical methods and follows traditional clinical decision-making processes. Methods Five ML models (Decision Tree, Random Forest, Support Vector Classifier, GradBoost, and a CNN) were trained on data collected from a large urban academic center to predict whether prolonged ICU stay would be required post-operatively. 535 patients who underwent posterior fusion or combined fusion for treatment of ASD were included in each model with a 70-20-10 train-test-validation split. Further analysis was performed using Shapley Additive Explanation (SHAP) values to provide insight into each model’s decision-making process. Results The model’s Area Under the Receiver Operating Curve (AUROC) ranged from 0.67 to 0.83. The Random Forest model achieved the highest score. The model considered length of surgery, complications, and estimated blood loss to be the greatest predictors of prolonged ICU stay based on SHAP values. Conclusions We developed a ML model that was able to predict whether prolonged ICU stay was required in ASD patients. Further SHAP analysis demonstrated our model aligned with traditional clinical thinking. Thus, ML models have strong potential to assist with risk stratification and more effective and cost-efficient care.

show abstract

Robust Prediction of Non-home Discharge After Thoracolumbar Spine Surgery With Ensemble Machine Learning and Validation on a Nationwide Cohort

Cited by 5 publications

References 13 publications

Endoscopic and Nonendoscopic Approaches to Single-Level Lumbar Spine Decompression: Propensity Score-Matched Comparative Analysis and Frailty-Driven Predictive Model

Endoscopic and Nonendoscopic Approaches to Single-Level Lumbar Spine Decompression: Propensity Score-Matched Comparative Analysis and Frailty-Driven Predictive Model

Limitations in Evaluating Machine Learning Models for Imbalanced Binary Outcome Classification in Spine Surgery: A Systematic Review

Explainable Machine Learning Approach to Prediction of Prolonged Intesive Care Unit Stay in Adult Spinal Deformity Patients: Machine Learning Outperforms Logistic Regression

Contact Info

Product

Resources

About