External Validation of PATHFx Version 3.0 in Patients Treated Surgically and Nonsurgically for Symptomatic Skeletal Metastases

Abstract

Background PATHFx is a clinical decision-support tool based on machine learning capable of estimating the likelihood of survival after surgery for patients with skeletal metastases. The applicability of any machine-learning tool depends not only on successful external validation in unique patient populations but also on remaining relevant as more effective systemic treatments are introduced. With advancements in the treatment of metastatic disease, it is our responsibility to patients to ensure clinical support tools remain contemporary and accurate. Question/purposes Therefore, we sought to (1) generate updated PATHFx models using recent data from patients treated at one large, urban tertiary referral center and (2) externally validate the models using two contemporary patient populations treated either surgically or nonsurgically with external-beam radiotherapy alone for symptomatic skeletal metastases for symptomatic lesions. Methods After obtaining institutional review board approval, we collected data on 208 patients undergoing surgical treatment for pathologic fractures at Memorial Sloan Kettering Cancer Center between 2015 and 2018. These data were combined with the original PATHFx training set (n = 189) to create the final training set (n = 397). We then created six Bayesian belief networks designed to estimate the likelihood of 1-month, 3-month, 6-month, 12-month, 18-month, and 24-month survival after treatment. Bayesian belief analysis is a statistical method that allows data-driven learning to arise from conditional probabilities by exploring relationships between variables to estimate the likelihood of an outcome using observed data. For external validation, we extracted the records of patients treated between 2016 and 2018 from the International Bone Metastasis Registry and records of patients treated nonoperatively with external-beam radiation therapy for symptomatic skeletal metastases from 2012 to 2016 using the Military Health System Data Repository (radiotherapy-only group). From each record, we collected the date of treatment, laboratory values at the time of treatment initiation, demographic data, details of diagnosis, and the date of death. All records reported sufficient follow-up to establish survival (yes/no) at 24-months after treatment. For external validation, we applied the data from each record to the new PATHFx models. We assessed calibration (calibration plots), accuracy (Brier score), discriminatory ability (area under the receiver operating characteristic curve [AUC]). Results The updated PATHFx version 3.0 models successfully classified survival at each time interval in both external validation sets and demonstrated appropriate discriminatory ability and model calibration. The Bayesian models were reasonably calibrated to the Memorial Sloan Kettering Cancer Center training set. External validation with 197 records from the International Bone Metastasis Registry and 192 records from the Military Health System Data Repository for analysis found Brier scores that were all less than 0.20, with upper bounds of the 95% confidence intervals all less than 0.25, both for the radiotherapy-only and International Bone Metastasis Registry groups. Additionally, AUC estimates were all greater than 0.70, with lower bounds of the 95% CI all greater than 0.68, except for the 1-month radiotherapy-only group. To complete external validation, decision curve analysis demonstrated clinical utility. This means it was better to use the PATHFx models when compared to the default assumption that all or no patients would survive at all time periods except for the 1-month models. We believe the favorable Brier scores (< 0.20) as well as DCA indicate these models are suitable for clinical use. Conclusions We successfully updated PATHFx using contemporary data from patients undergoing either surgical or nonsurgical treatment for symptomatic skeletal metastases. These models have been incorporated for clinical use on PATHFx version 3.0 (https://www.pathfx.org). Clinically, external validation suggests it is better to use PATHFx version 3.0 for all time periods except when deciding whether to give radiotherapy to patients with the life expectancy of less than 1 month. This is partly because most patients survived 1-month after treatment. With the advancement of medical technology in treatment and diagnosis for patients with metastatic bone disease, part of our fiduciary responsibility is to the main current clinical support tools. Level of Evidence Level III, therapeutic study.

Prosthesis or osteosynthesis for the treatment of a pathological hip fracture? A nationwide registry-based cohort study

“…This is contrary to the traumatic hip fracture scenario, where total hip arthroplasty is associated with better functional outcome [18] . Prognostication tools allow for choice of patients with MBD who are expected to have longer survival and thus benefit from total hip arthroplasty [19] , [20] .…”

Does Surgical Technique Influence the Burden of Lung Metastases in Patients with Pathologic Long Bone Fractures?

“…PATHFx) using techniques such as machine learning requires accurate inputs of multiple factors predicting patient outcome. [9,10] By furthering our understanding of surgical technical factors that are related to overall patient survival, we can more accurately counsel patients on treatment goals, and modify surgical techniques accordingly. Interestingly, while surgical technique was not statistically associated with metastatic progression in the lung in this study, we did demonstrate that those who underwent IMN xation experience a poorer prognosis than those undergoing reconstruction with either arthroplasty or ORIF techniques.…”

Current Overview of Treatment for Metastatic Bone Disease

“…The model includes both objective quantifiable variables (age, sex, primary type, Eastern Cooperative Oncology Group performance status score, presence of visceral metastases, presence of multiple skeletal metastases, pathological fracture, hemoglobin, and lymphocyte count), and subjective variables (surgeon’s estimate of survival), although it has been reported that prognosis can be accurately predicted without this subjective variable [ 9 ]. PathFx has been externally validated in many different patient populations [ 9 , 10 , 11 , 12 ] and has been recently updated to PathFx version 3.0 ( accessed on 10 July 2021) [ 13 ]. Meares et al [ 14 ] compared several models, including the revised Katagiri model [ 15 ], SSG score [ 16 ], Janssen nomogram [ 17 ], and SPRING 13 nomogram [ 18 ], and reported that OPTModel demonstrated the highest accuracy at predicting 12-month (area under the curve [AUC] = 0.79) and 24-month survival (AUC = 0.77) after surgical management, while PathFx was the most accurate at predicting 3-month (AUC = 0.70) and 6-month survival (AUC = 0.70).…”

“…The final model was incorporated into a freely accessible web application available at accessed on 10 July 2021 [ 19 ]. Table 3 summarizes the prognosis prediction model created by machine learning [ 9 , 10 , 11 , 12 , 13 , 14 , 19 , 20 , 21 , 22 , 23 ]. Errani et al analyzed 159 patients with bone metastases in the extremities who underwent surgery [ 24 ] and reported that pathological CRP (≥1.0 mg/dL) and primary tumor diagnosis were significant negative prognostic factors at 12-month survival.…”