The preoperative machine learning algorithm for extremity metastatic disease can predict 90‐day and 1‐year survival: An external validation study

Abstract

Background:The prediction of survival is valuable to optimize treatment of metastatic long-bone disease. The Skeletal Oncology Research Group (SORG) machine-learning (ML) algorithm has been previously developed and internally validated. The purpose of this study was to determine if the SORG ML algorithm accurately predicts 90-day and 1-year survival in an external metastatic long-bone disease patient cohort.Methods: A retrospective review of 264 patients who underwent surgery for long-bone metastases between 2003 and 2019 was performed. Variables used in the stochastic gradient boosting SORG algorithm were age, sex, primary tumor type, visceral/brain metastases, systemic therapy, and 10 preoperative laboratory values. Model performance was calculated by discrimination, calibration, and overall performance.Results: The SORG ML algorithms retained good discriminative ability (area under the cure [AUC]: 0.83; 95% confidence interval [CI]: 0.76-0.88 for 90-day mortality and AUC: 0.84; 95% CI: 0.79-0.88 for 1-year mortality), calibration, overall performance, and decision curve analysis. Conclusion:The previously developed ML algorithms demonstrated good performance in the current study, thereby providing external validation. The models were incorporated into an accessible application (https://sorg-apps.shinyapps.io/ extremitymetssurvival/) that may be freely utilized by clinicians in helping predict survival for individual patients and assist in informative decision-making discussion before operative management of long bone metastatic lesions.

Comparison of eight modern preoperative scoring systems for survival prediction in patients with extremity metastasis

“…In statistics, using more variables to predict an outcome will render better discriminatory results. Although in theory including more variables could also introduce the risk of overfitting and decrease the model's generalizability in newer, independent datasets, calibration analyses in this or other SORG‐MLA validation studies did not observe this issue 19,27,56 . Our results indicated SORG‐MLA was a reliable survival prediction tool for Han Chinese patients with skeletal metastasis.…”

“…Although in theory including more variables could also introduce the risk of overfitting and decrease the model's generalizability in newer, independent datasets, calibration analyses in this or other SORG-MLA validation studies did not observe this issue. 19,27,56 Our results indicated SORG-MLA was a reliable survival prediction tool for Han Chinese patients with skeletal metastasis. Some clinicians might not be well versed in statistical jargons like discrimination, calibration, and Brier score.…”

What’s New in Musculoskeletal Tumor Surgery

“…Predictive models and clinical risk assessment tools continue to be an area of active research in metastatic bone disease 67,[73][74][75][76] . The Skeletal Oncology Research Group (SORG) machinelearning (ML) algorithms have been widely studied as tools to estimate survival in patients with extremity metastases.…”

“…Table III shows some recent studies evaluating patients with metastatic bone disease [66][67][68][69][70][71][72][73][74][75][76] .…”

“…The Skeletal Oncology Research Group (SORG) machine-learning (ML) algorithms have been widely studied as tools to estimate survival in patients with extremity metastases. The SORG ML algorithms have now been validated in different external cohorts and have demonstrated good discriminatory validity in different patient populations 67,75 . Raad et al utilized the National Surgical Quality Improvement Program (NSQIP) database to create a clinical tool to predict 30-day mortality in patients undergoing surgical fixation of metastatic extremity fractures.…”