A Review of Survival Analysis Methods Used in NICE Technology Appraisals of Cancer Treatments: Consistency, Limitations, and Areas for Improvement

Abstract

Objectives. In June 2011, the National Institute for Health and Care Excellence (NICE) Decision Support Unit published a Technical Support Document (TSD) providing recommendations on survival analysis for NICE technology appraisals (TAs). Survival analysis outputs are influential inputs into economic models estimating the cost-effectiveness of new cancer treatments. Hence, it is important that systematic and justifiable model selection approaches are used. This study investigates the extent to which the TSD recommendations have been followed since its publication. Methods. We reviewed NICE cancer TAs completed between July 2011 and July 2017. Information on survival analyses undertaken and associated critiques for overall survival (OS) and progression-free survival were extracted from the company submissions, Evidence Review Group (ERG) reports, and final appraisal determination documents. Results. Information was extracted from 58 TAs. Only 4 (7%) followed all TSD recommendations for OS outcomes. The vast majority (91%) compared a range of common parametric models and assessed their fit to the data (86%). Only a minority of TAs included an assessment of the shape of the hazard function (38%) or proportional hazards assumption (40%). Validation of the extrapolated portion of the survival function using external data was attempted in a minority of TAs (40%). Extrapolated survival functions were frequently criticized by ERGs (71%). Conclusions. Survival analysis within NICE TAs remains suboptimal, despite publication of the TSD. Model selection is not undertaken in a systematic way, resulting in inconsistencies between TAs. More attention needs to be given to assessing hazard functions and validation of extrapolated survival functions. Novel methods not described in the TSD have been used, particularly in the context of immuno-oncology, suggesting that an updated TSD may be of value.

Extrapolating Parametric Survival Models in Health Technology Assessment: A Simulation Study

“…It is clear that reliance solely on statistical measures of goodness of fit can result in severe bias and incorrect estimation of RMST even when 2 or more measures might agree on the optimal model, and it is reassuring that model plausibility is also usually considered in practice, although not always. 2 Our scenarios of trial followup suggested that the models preferred by the selection methods were often associated with significantly higher MSEs than the best parametric model, suggesting there is room for improvement over AIC and BIC when selecting models for extrapolation. We have shown that the difference in the underlying assumptions of the shapes of the models is an important consideration, given that the log-normal, log-logistic, and Gompertz models often provided markedly difference estimates of RMST compared to the other parametric forms, particularly in the scenarios of trial follow-up.…”

“…The most common approach to extrapolation in NICE appraisals is to select a single model from a range of parametric models (e.g., exponential). 1,2 Candidate models are usually compared on 2 metrics: on the goodness of fit to the observed data and on the plausibility of the extrapolations. This article considers the former, since the plausibility of extrapolations can only objectively be demonstrated when suitably mature data are available.…”

Onwards and Upwards: A Systematic Survey of Economic Evaluation Methods in Oncology

“…In addition, the analytical methods selected for economic evaluation in oncology have been shown to influence survival results [10], and therefore it is important for appropriate methods to be used when evaluating oncology products. However, only two previous studies have detailed the use of survival analysis in economic evaluations in oncology [11,12]. The first of these studies examined survival modeling and extrapolation techniques used in oncology submissions to the NICE in the UK before and after publication of the NICE Decision Support Unit's Technical Support Document (TSD) on survival analysis [13].…”

“…A second study from 2019 reviewed 58 NICE technology appraisals and examined the extent to which recommendations made in the NICE Decision Support Unit's TSD on survival analysis have been followed since its publication [11]. The authors found that while there were increases in validation of the results using data and/or clinical opinion following publication of the TSD, the proportion of submissions that adhered to the TSD recommendations did not change substantively over time.…”

Extrapolating Parametric Survival Models in Health Technology Assessment Using Model Averaging: A Simulation Study

“…Time-to-event outcomes are routinely modeled in health technology appraisals by fitting parametric models to observed data and extrapolating until no patients are predicted to remain at risk of an event. 1,2 These models are commonly used to predict the amount of time patients spend in particular health states, either through the estimation of transition probabilities or mean survival times, depending on the type of economic model. The time in each health state is combined with a health-related quality-of-life utility value and the costs of factors including therapy, disease management, and adverse events to produce an overall estimate of the cost-effectiveness of a treatment relative to a comparator.…”