What Did Time Tell Us? A Comparison and Retrospective Validation of Different Survival Extrapolation Methods for Immuno-Oncologic Therapy in Advanced or Metastatic Renal Cell Carcinoma

Klijn, S.; Fenwick, Elisabeth; Kroep, Sonja; Johannesen, Kasper; Malcolm, Bill; Kurt, Murat; Kiff, Christopher; Borrill, J.

doi:10.1007/s40273-020-00989-1

Cited by 17 publications

(18 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The design of this study was informed by previous research that investigated the predictive accuracy of standard versus more flexible extrapolation methods used to evaluate the long-term benefit of immune checkpoint inhibitors. 7 – 10 The main aim of this study was to retrospectively assess the performance of 6 survival extrapolation methods fitted to extended follow-up data from CheckMate 067, a phase III randomized controlled trial that compared the use of the immune checkpoint inhibitor combination nivolumab (a programmed death 1 checkpoint inhibitor) plus ipilimumab (an anti–cytotoxic T-cell lymphocyte-antigen 4 checkpoint inhibitor) or nivolumab alone with ipilimumab alone in previously untreated patients with advanced melanoma. 11 Successive artificial data cuts with varying lengths of follow-up were created to assess how the predictive accuracy of the methods may vary based on the maturity of the data.…”

Section: Introductionmentioning

confidence: 99%

Heterogeneity in Survival with Immune Checkpoint Inhibitors and Its Implications for Survival Extrapolations: A Case Study in Advanced Melanoma

Paly

Kurt

Zhang

et al. 2022

MDM Policy & Practice

Self Cite

View full text Add to dashboard Cite

Background Survival heterogeneity and limited trial follow-up present challenges for estimating lifetime benefits of oncology therapies. This study used CheckMate 067 (NCT01844505) extended follow-up data to assess the predictive accuracy of standard parametric and flexible models in estimating the long-term overall survival benefit of nivolumab plus ipilimumab (an immune checkpoint inhibitor combination) in advanced melanoma. Methods Six sets of survival models (standard parametric, piecewise, cubic spline, mixture cure, parametric mixture, and landmark response models) were independently fitted to overall survival data for treatments in CheckMate 067 (nivolumab plus ipilimumab, nivolumab, and ipilimumab) using successive data cuts (28, 40, 52, and 60 mo). Standard parametric models allow survival extrapolation in the absence of a complex hazard. Piecewise and cubic spline models allow additional flexibility in fitting the hazard function. Mixture cure, parametric mixture, and landmark response models provide flexibility by explicitly incorporating survival heterogeneity. Sixty-month follow-up data, external ipilimumab data, and clinical expert opinion were used to evaluate model estimation accuracy. Lifetime survival projections were compared using a 5% discount rate. Results Standard parametric, piecewise, and cubic spline models underestimated overall survival at 60 mo for the 28-mo data cut. Compared with other models, mixture cure, parametric mixture, and landmark response models provided more accurate long-term overall survival estimates versus external data, higher mean survival benefit over 20 y for the 28-mo data cut, and more consistent 20-y mean overall survival estimates across data cuts. Conclusion This case study demonstrates that survival models explicitly incorporating survival heterogeneity showed greater accuracy for early data cuts than standard parametric models did, consistent with similar immune checkpoint inhibitor survival validation studies in advanced melanoma. Research is required to assess generalizability to other tumors and disease stages. Highlights Given that short clinical trial follow-up periods and survival heterogeneity introduce uncertainty in the health technology assessment of oncology therapies, this study evaluated the suitability of conventional parametric survival modeling approaches as compared with more flexible models in the context of immune checkpoint inhibitors that have the potential to provide lasting survival beneﬁts. This study used extended follow-up data from the phase III CheckMate 067 trial (NCT01844505) to assess the predictive accuracy of standard parametric models in comparison with more flexible methods for estimating the long-term survival benefit of the immune checkpoint inhibitor combination of nivolumab plus ipilimumab in advanced melanoma. Mixture cure, parametric mixture, and landmark response models provided more accurate estimates of long-term overall survival versus external data than other models tested. In this case study with immune checkpoint inhibitor therapies in advanced melanoma, extrapolation models that explicitly incorporate differences in cancer survival between observed or latent subgroups showed greater accuracy with both early and later data cuts than other approaches did.

show abstract

Section: Introductionmentioning

confidence: 99%

Heterogeneity in Survival with Immune Checkpoint Inhibitors and Its Implications for Survival Extrapolations: A Case Study in Advanced Melanoma

Paly

Kurt

Zhang

et al. 2022

MDM Policy & Practice

Self Cite

View full text Add to dashboard Cite

show abstract

“…They concluded that models with nonmonotonic hazards were consistently associated with better statistical fit and more accurate prediction of long-term survival for ICI monotherapies, consistent with the findings of the present study (62). Finally, Klijn et al have compared the accuracy over time of a range of extrapolation methods to predict OS of patient treated with nivolumab based on IPD from CheckMate 025 study (63). All extrapolation methods underestimated long-term OS compared to observed data with the exception of mixture models.…”

Section: Discussionmentioning

confidence: 99%

“…Finally, it would also be interesting to compare the performance of the selected models in assessment reports from other HTA agencies. For example, agencies in England and Wales, Sweden, and Norway have used different models to estimate long-term survival following treatment of RCC with nivolumab, although the data source was identical (CheckMate 025) (63).…”

Section: Discussionmentioning

confidence: 99%

A Review of Overall Survival Extrapolations of Immune-Checkpoint Inhibitors Used in Health Technology Assessments by the French Health Authorities

Grumberg

Roze²,

Chevalier³

et al. 2022

Int J Technol Assess Health Care

Self Cite

View full text Add to dashboard Cite

Objectives Extrapolation is often required to inform cost-effectiveness (CE) evaluations of immune-checkpoint inhibitors (ICIs) since survival data from pivotal clinical trials are seldom complete. The objectives of this study were to evaluate the accuracy of estimates of long-term overall survival (OS) predicted in French CE assessment reports of ICIs, and to identify models presenting the best fit to the observed long-term survival data. Methods A systematic review of French assessment reports of ICIs in the metastatic setting since inception until May 2020 was performed. A targeted literature review was conducted to collect associated extended follow-up of randomized controlled trials (RCTs) used in the CE assessment reports. Difference between projected and observed OS was calculated. A range of standard parametric and spline-based models were applied to the extended follow-up data from the RCT to determine the best-fitting survival models. Results Of the 121 CE assessment reports published, 11 reports met the inclusion criteria. OS was underestimated in 73 percent of the CE assessment reports. The mean relative difference between each source was −13 percent (median: −15 percent; IQR: −0.4 to 26 percent). Models providing the best fit were those that could reflect nonmonotonic hazards. Conclusions Based on the available data at the time of submission, longer-term survival of ICIs was not fully captured by the extrapolation models used in CE assessments. Standard and flexible parametric models which can capture nonmonotonic hazard functions provided the best fit to the extended follow-up data. However, these models may still have performed poorly if fitted to survival data available at the time of submission to the French National Authority for Health.

show abstract

“…If heterogeneity is suspected, but not detected or attributable to any known covariate, fitting separate models for different subgroups is not an option. It is possible that flexible parametric approaches [ 13 ] or mixture cure models [ 14 ] might better capture the heterogeneity than would traditional parametric approaches; however, these were beyond the scope of this study, and further investigation is needed. Data appearing to follow a complex hazard rate may be a consequence of a heterogeneous population containing subgroups that each have a much simpler underlying hazard rate.…”

Section: Discussionmentioning

confidence: 99%

Biased Survival Predictions When Appraising Health Technologies in Heterogeneous Populations

2021

View full text Add to dashboard Cite

Introduction Time-to-event data from clinical trials are routinely extrapolated using parametric models to estimate the cost effectiveness of novel therapies, but how this approach performs in the presence of heterogeneous populations remains unknown. Methods We performed a simulation study of seven scenarios with varying exponential distributions modelling treatment and prognostic effects across subgroup and complement populations, with follow-up typical of clinical trials used to appraise the cost effectiveness of therapies by agencies such as the UK National Institute for Health and Care Excellence (NICE). We compared established and emerging methods of estimating population life-years (LYs) using parametric models. We also proved analytically that an exponential model fitted to censored heterogeneous survival times sampled from two distinct exponential distributions will produce a biased estimate of the hazard rate and LYs. Results LYs are underestimated by the methods in the presence of heterogeneity, resulting in either under-or overestimation of the incremental benefit. In scenarios where the overestimation of benefit is likely, which is of interest to the healthcare provider, the method of taking the average LYs from all plausible models has the least bias. LY estimates from complete Kaplan-Meier curves have high variation, suggesting mature data may not be a reliable solution. We explore the effect of increasing trial sample size and accounting for detected treatment-subgroup interactions. Conclusions The bias associated with heterogeneous populations suggests that NICE may need to be more cautious when appraising therapies and to consider model averaging or the separate modelling of subgroups when heterogeneity is suspected or detected.

show abstract

What Did Time Tell Us? A Comparison and Retrospective Validation of Different Survival Extrapolation Methods for Immuno-Oncologic Therapy in Advanced or Metastatic Renal Cell Carcinoma

Cited by 17 publications

References 24 publications

Heterogeneity in Survival with Immune Checkpoint Inhibitors and Its Implications for Survival Extrapolations: A Case Study in Advanced Melanoma

Heterogeneity in Survival with Immune Checkpoint Inhibitors and Its Implications for Survival Extrapolations: A Case Study in Advanced Melanoma

A Review of Overall Survival Extrapolations of Immune-Checkpoint Inhibitors Used in Health Technology Assessments by the French Health Authorities

Biased Survival Predictions When Appraising Health Technologies in Heterogeneous Populations

Contact Info

Product

Resources

About