Detailed statistical assessment of the characteristics of the ESMO Magnitude of Clinical Benefit Scale (ESMO-MCBS) threshold rules

Abstract

BackgroundThe European Society for Medical Oncology (ESMO) has developed the ESMO Magnitude of Clinical Benefit Scale (ESMO-MCBS), a tool to assess the magnitude of clinical benefit from new cancer therapies. Grading is guided by a dual rule comparing the relative benefit (RB) and the absolute benefit (AB) achieved by the therapy to prespecified threshold values. The ESMO-MCBS v1.0 dual rule evaluates the RB of an experimental treatment based on the lower limit of the 95%CI (LL95%CI) for the hazard ratio (HR) along with an AB threshold. This dual rule addresses two goals: inclusiveness: not unfairly penalising experimental treatments from trials designed with adequate power targeting clinically meaningful relative benefit; and discernment: penalising trials designed to detect a small inconsequential benefit.MethodsBased on 50 000 simulations of plausible trial scenarios, the sensitivity and specificity of the LL95%CI rule and the ESMO-MCBS dual rule, the robustness of their characteristics for reasonable power and range of targeted and true HRs, are examined. The per cent acceptance of maximal preliminary grade is compared with other dual rules based on point estimate (PE) thresholds for RB.ResultsFor particularly small or particularly large studies, the observed benefit needs to be relatively big for the ESMO-MCBS dual rule to be satisfied and the maximal grade awarded. Compared with approaches that evaluate RB using the PE thresholds, simulations demonstrate that the MCBS approach better exhibits the desired behaviour achieving the goals of both inclusiveness and discernment.ConclusionsRB assessment using the LL95%CI for HR rather than a PE threshold has two advantages: it diminishes the probability of excluding big benefit positive studies from achieving due credit and, when combined with the AB assessment, it increases the probability of downgrading a trial with a statistically significant but clinically insignificant observed benefit.

Monitoring evidence on overall survival benefits of anticancer drugs approved by the European Medicines Agency between 2009 and 2015

“…To evaluate the clinical benefit of anticancer drugs, both relative-benefits (HR) and absolute-benefits (median) in OS are commonly reported [25,26]. Other survival parameters, especially in cases where a large benefit for a few patients is expected, are absolute and proportional gains in the long-term survival rates [26].…”

EHA evaluation of the ESMO—Magnitude of Clinical Benefit Scale version 1.1 (ESMO-MCBS v1.1) for haematological malignancies

“…Primary or secondary endpoints included in the scoring system are OS, progression-free survival (PFS), QOL, treatment toxicity or response rates. In developing the ESMO-MCBS scale, ESMO aspired to meet standards for ‘accountability for reasonableness’,5 6 incorporating extensive field testing, statistical modelling7 and peer review of the ‘reasonableness’ of the generated results into the development process. The ESMO-MCBS is currently incorporated in ESMO’s clinical practice guidelines and is being used as part of HTA processes 8 9…”

Systemic therapy for early-stage breast cancer: learning from the past to build the future

“…With the exponential increase in cancer drug costs in the past years, agreeing on a ‘prioritization’ list based on the real added value of such drugs is a legitimate response. This much-needed but difficult exercise has been conducted by ASCO, with the development of the ASCO Value Framework 91 , and by ESMO with the development of the ESMO Magnitude of Clinical Benefit Scale (MCBS) 92 , which values improvements in OS and/or quality of life far more heavily than improvements in ORR or extension of PFS. Interestingly, DFS has been the subject of an intense debate, leading to a reasonable consensus: living additional months or years without detectable disease has greater value than living somewhat longer with stable, non-progressive disease; thus, improvements in median DFS of sufficient magnitude are credited with high scores on the ESMO-MCBS until OS data become available, at which point a lack of OS benefit induces a one-point downgrade (ESMO-MCBS version 2.0 will soon be published 93 ).…”