Developing an Improved Statistical Approach for Survival Estimation in Bone Metastases Management: The Bone Metastases Ensemble Trees for Survival (BMETS) Model

Alcorn, Sara R.; Fiksel, Jacob; Wright, Jean L.; Elledge, C.R.; Smith, Thomas J.; Perng, Powell; Saleemi, Sarah; McNutt, Todd; DeWeese, Theodore L.; Zeger, Scott L.

doi:10.1016/j.ijrobp.2020.05.023

Cited by 22 publications

(30 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As per methodology previously described, 26 27 potential predictor covariates were identified from the literature and used to build the BMETS survival prediction model. Stakeholder feedback universally cited uncertainty with survival estimates as the primary source of decisional conflict when selecting appropriate palliative RT regimens.…”

Section: Resultsmentioning

confidence: 99%

“…As previously described during model development, 26 values of covariates entered that inform the BMETS survival prediction model are used to create a survival curve. The resultant survival curve is interactive, displaying probability of survival at various times from consultation.…”

Section: Resultsmentioning

confidence: 99%

“…As per the ODSF, the goal of the BMETS‐DSP was to address modifiable determinants of the decision, particularly those that contribute to decisional conflict and uncertainty 31 . From past literature 34 and work developing the BMETS model, 26 we assumed that survival prediction would be a central theme and planned to use the BMETS model for this purpose in the decision support tool. To uphold delivery of standard‐of‐care and evidence‐based medicine, we elected to provide educational material of the highest category of evidence available, with clear citations to enable review of the source data by users if desired.…”

Section: Methodsmentioning

confidence: 99%

“…To promote selection of palliative RT regimens more closely aligned with predicted patient prognosis and best evidence in the management of symptomatic bone metastases, we sought to develop the provider‐facing BMETS Bone Metastases Ensemble Trees for Survival‐Decision Support Platform (BMETS‐DSP) for use in this patient population. We aimed to design a tool that (1) collects patient‐specific demographic, disease and treatment data, (2) displays an individualised survival curve based on the prescribed BMETS machine learning survival prediction model 26 and (3) provides case‐specific, evidence‐based recommendations for RT and alternative interventions to aid in the decision‐making process. Here, we describe the underlying conceptual framework, platform components, and steps in the development of the BMETS‐DSP.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Improving providers' survival estimates and selection of prognosis‐ and guidelines‐appropriate treatment for patients with symptomatic bone metastases: Development of the Bone Metastases Ensemble Trees for Survival Decision Support Platform

Alcorn

Elledge

LaVigne

et al. 2022

Evaluation Clinical Practice

Self Cite

View full text Add to dashboard Cite

Rationale, Aims and Objectives: In the management of symptomatic bone metastases, current practice guidelines do not provide clear methodology for selecting palliative radiotherapy (RT) regimens based on specific patient and disease features. Decision support aids may offer an effective means for translating the complex data needed to render individualised treatment decisions, yet no such tools are available for use in this setting. Thus, we describe the development of the Bone Metastases Ensemble Trees for Survival-Decision Support Platform (BMETS-DSP), which aims to optimise selection of evidence-based, individualised palliative RT regimens. Method: The Ottawa Decision Support Framework was used as the theoretical basis for development of BMETS-DSP. First, we utilised stakeholder input and review of the literature to assess determinants underlying the provider decision. Based on this assessment and iterative stakeholder feedback, we developed the web-based, provider-facing BMETS-DSP. Consistent with the underlying theoretical framework, our design also included assessment of decision quality using the International Patient Decision Aids Standards (IPDAS) certification checklist. Results: Stakeholder input and review of 54 evidence-based publications identified the following determinants of the provider decision: estimated prognosis, characteristics of the target symptomatic lesion and the primary cancer type, consideration of alternative interventions, access to patient-specific recommendations, and patient preferences. Based on these determinants, we developed the BMETS-DSP that (1) collects patient-specific data, (2) displays an individualised predicted survival curve, and (3) provides case-specific, evidence-based recommendations regarding RT, open surgery, systemic therapy, and hospice referral to aid in the

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Improving providers' survival estimates and selection of prognosis‐ and guidelines‐appropriate treatment for patients with symptomatic bone metastases: Development of the Bone Metastases Ensemble Trees for Survival Decision Support Platform

Alcorn

Elledge

LaVigne

et al. 2022

Evaluation Clinical Practice

Self Cite

View full text Add to dashboard Cite

show abstract

“…11 We chose to frame information by delivering social comparison feedback, given its low cost and proven success in other clinical contexts. [33][34][35] Although we met our primary outcome, we suspect a greater effect size would be seen by guiding choice through default options (automatic hospice informational visit at a defined time point, such as 6 months estimated by the surprise question 42 or more sophisticated models), 43 enhancing choice (allowing concurrent care), and use of an ongoing (multiple peer comparison reports) versus a single intervention. Much can be learned from concurrent care implementation in Veterans Affairs (VA) medical centers where 68% of patients with cancer use hospice, with 25% using concurrent care.…”

Section: Discussionmentioning

confidence: 99%

Oncologist Peer Comparisons as a Behavioral Science Strategy to Improve Hospice Utilization

Sedhom

Blackford

Gupta

et al. 2022

JCO Oncology Practice

Self Cite

View full text Add to dashboard Cite

PURPOSE: Hospice utilization metrics are essential for any serious effort to improve end-of-life care in oncology. However, oncologists do not routinely receive these personalized reports. We evaluated whether a behavioral science intervention, using peer comparisons coupled with social norms, was associated with improvements in hospice use. METHODS: Oncologists at two academic practices of Johns Hopkins Medicine were randomly assigned to receive a peer comparison report by e-mail displaying individual hospice utilization metrics compared with top-performing peers or to receive no report. The data accrued for the intervention represented hospice utilization for the previous calendar year. The intervention period was from June 1, 2020, to December 30, 2020, and included oncologists from both the solid and hematologic malignancies programs. The primary outcome was the proportion of patients between groups with short hospice length of stay (LOS; defined as ≤ 7 days) after 6 months. Secondary outcomes included hospice referral rate, enrollment rate, and median LOS. RESULTS: Forty-seven oncologists participated. The percent of patients with a short hospice stay in the intervention group was lower (17.4%) compared with patients treated by physicians in the usual care group (46.3%, difference = 21.8%; 95% CI, 16.0 to 41.6; P < .001). Receipt of peer comparisons was associated with a greater likelihood of enrolling in hospice (73.7% v 42.8%; difference = 31.1%; 95% CI, 20.4 to 41.7; P < .001) and a longer hospice LOS (37.2 v 18.3 days; difference = 17.2; 95% CI, 8.8 to 25.7 days; P < .001). CONCLUSION: Peer comparisons improved hospice utilization metrics among a group of academic oncologists. Behavioral science offers one pragmatic strategy to overcome suboptimal oncologist decision-making biases related to hospice use.

show abstract

Deep learning‐based survival analysis for brain metastasis patients with the national cancer database

Bice

Kirby

Bahr

et al. 2020

J Applied Clin Med Phys

View full text Add to dashboard Cite

Purpose Prognostic indices such as the Brain Metastasis Graded Prognostic Assessment have been used in clinical settings to aid physicians and patients in determining an appropriate treatment regimen. These indices are derivative of traditional survival analysis techniques such as Cox proportional hazards (CPH) and recursive partitioning analysis (RPA). Previous studies have shown that by evaluating CPH risk with a nonlinear deep neural network, DeepSurv, patient survival can be modeled more accurately. In this work, we apply DeepSurv to a test case: breast cancer patients with brain metastases who have received stereotactic radiosurgery. Methods Survival times, censorship status, and 27 covariates including age, staging information, and hormone receptor status were provided for 1673 patients by the NCDB. Monte Carlo cross‐validation with 50 samples of 1400 patients was used to train and validate the DeepSurv, CPH, and RPA models independently. DeepSurv was implemented with L2 regularization, batch normalization, dropout, Nesterov momentum, and learning rate decay. RPA was implemented as a random survival forest (RSF). Concordance indices of test sets of 140 patients were used for each sample to assess the generalizable predictive capacity of each model. Results Following hyperparameter tuning, DeepSurv was trained at 32 min per sample on a 1.33 GHz quad‐core CPU. Test set concordance indices of 0.7488 ± 0.0049, 0.6251 ± 0.0047, and 0.7368 ± 0.0047, were found for DeepSurv, CPH, and RSF, respectively. A Tukey HSD test demonstrates a statistically significant difference between the mean concordance indices of the three models. Conclusion Our results suggest that deep learning‐based survival prediction can outperform traditional models, specifically in a case where an accurate prognosis is highly clinically relevant. We recommend that where appropriate data are available, deep learning‐based prognostic indicators should be used to supplement classical statistics.

show abstract

Developing an Improved Statistical Approach for Survival Estimation in Bone Metastases Management: The Bone Metastases Ensemble Trees for Survival (BMETS) Model

Cited by 22 publications

References 33 publications

Improving providers' survival estimates and selection of prognosis‐ and guidelines‐appropriate treatment for patients with symptomatic bone metastases: Development of the Bone Metastases Ensemble Trees for Survival Decision Support Platform

Improving providers' survival estimates and selection of prognosis‐ and guidelines‐appropriate treatment for patients with symptomatic bone metastases: Development of the Bone Metastases Ensemble Trees for Survival Decision Support Platform

Oncologist Peer Comparisons as a Behavioral Science Strategy to Improve Hospice Utilization

Deep learning‐based survival analysis for brain metastasis patients with the national cancer database

Contact Info

Product

Resources

About