Can a Bayesian belief network for survival prediction in patients with extremity metastases (PATHFx) be externally validated in an Asian cohort of 356 surgically treated patients?

Abstract

Background and purpose: Predicted survival may influence the treatment decision for patients with skeletal extremity metastasis, and PATHFx was designed to predict the likelihood of a patient dying in the next 24 months. However, the performance of prediction models could have ethnogeographical variations. We asked if PATHFx generalized well to our Taiwanese cohort consisting of 356 surgically treated patients with extremity metastasis.Patients and methods: We included 356 patients who underwent surgery for skeletal extremity metastasis in a tertiary center in Taiwan between 2014 and 2019 to validate PATHFx’s survival predictions at 6 different time points. Model performance was assessed by concordance index (c-index), calibration analysis, decision curve analysis (DCA), Brier score, and model consistency (MC).Results: The c-indexes for the 1-, 3-, 6-, 12-, 18-, and 24-month survival estimations were 0.71, 0.66, 0.65, 0.69, 0.68, and 0.67, respectively. The calibration analysis demonstrated positive calibration intercepts for survival predictions at all 6 timepoints, indicating PATHFx tended to underestimate the actual survival. The Brier scores for the 6 models were all less than their respective null model’s. DCA demonstrated that only the 6-, 12-, 18-, and 24-month predictions appeared useful for clinical decision-making across a wide range of threshold probabilities. The MC was < 0.9 when the 6- and 12-month models were compared with the 12-month and 18-month models, respectively.Interpretation: In this Asian cohort, PATHFx’s performance was not as encouraging as those of prior validation studies. Clinicians should be cognizant of the potential decline in validity of any tools designed using data outside their particular patient population. Developers of survival prediction tools such as PATHFx might refine their algorithms using data from diverse, contemporary patients that is more reflective of the world’s population.

Comparison of survival prediction models for bone metastases of the extremities following surgery

“…Estimation of life expectancy plays a central role in deciding the treatment of BM-E with impending or actual pathological fractures. Multiple postoperative survival prediction models have been externally validated for their predictive performance, 3,5,12,13,24,25 yet direct comparison of the models in an Asian population was missing. Our study is the first to perform comparative analysis of the postoperative survival prediction models for BM-E in an Asian cohort.…”

“…External validation of individual survival prediction models of bone metastases have been performed in Asian populations, showing generalizability to the ethnic group. [3][4][5][6][7] Comparative analyses of survival prediction models of BM-E have been reported in Australian, Chilean, and Italian cohorts, yet did not include a few key statistical analyses (calibration curve and decision curve analysis). [8][9][10] Previous literature has narrowed candidates for externally validated survival prediction models with good predictive performance.…”

A Machine Learning Algorithm for Predicting 6-Week Survival in Spinal Metastasis: An External Validation Study Using 2,768 Taiwanese Patients

“…26 MC was introduced recently due to the observation of paradoxical survival prediction results. 7,17,27,28 To follow human intuition, survival probability in the near future should be higher than that in the distant future since death is considered irreversible. It is even so when the models use the same parameters for prediction.…”

Psoas muscle area is an independent survival prognosticator in patients undergoing surgery for long‐bone metastases

“…PATHFx, a Bayesian belief network-based algorithm developed in 2011, has been validated in multiple regions such as the United States, Japan, Italy, the Scandinavian peninsula, and Taiwan. 10 It is currently in its third iteration as an online application for clinical use. 11 SORG-MLA was introduced in 2020 as a web-based application that incorporated 15 prognostic factors and has been shown to achieve great discrimination on both internal and external validations.…”

“…More recently, two machine learning‐based systems were introduced: PATHFx and the Skeletal Oncology Research Group Machine Learning Algorithm (SORG‐MLA). PATHFx, a Bayesian belief network‐based algorithm developed in 2011, has been validated in multiple regions such as the United States, Japan, Italy, the Scandinavian peninsula, and Taiwan 10 . It is currently in its third iteration as an online application for clinical use 11 .…”