Biases in study design, implementation, and data analysis that distort the appraisal of clinical benefit and ESMO-Magnitude of Clinical Benefit Scale (ESMO-MCBS) scoring

Abstract

Background: The European Society for Medical Oncology-Magnitude of Clinical Benefit Scale (ESMO-MCBS) is a validated, widely used tool developed to score the clinical benefit from cancer medicines reported in clinical trials. ESMO-MCBS scores assume valid research methodologies and quality trial implementation. Studies incorporating flawed design, implementation, or data analysis may generate outcomes that exaggerate true benefit and are not generalisable. Failure to either indicate or penalise studies with bias undermines the intention and diminishes the integrity of ESMO-MCBS scores. This review aimed to evaluate the adequacy of the ESMO-MCBS to address bias generated by flawed design, implementation, or data analysis and identify shortcomings in need of amendment. Methods: As part of a refinement of the ESMO-MCBS, we reviewed trial design, implementation, and data analysis issues that could bias the results. For each issue of concern, we reviewed the ESMO-MCBS v1.1 approach against standards derived from Helsinki guidelines for ethical human research and guidelines from the International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use, the Food and Drugs Administration, the European Medicines Agency, and European Network for Health Technology Assessment. Results: Six design, two implementation, and two data analysis and interpretation issues were evaluated and in three, the ESMO-MCBS provided adequate protections. Seven shortcomings in the ability of the ESMO-MCBS to identify and address bias were identified. These related to (i) evaluation of the control arm, (ii) crossover issues, (iii) criteria for noninferiority, (iv) substandard post-progression treatment, (v) post hoc subgroup findings based on biomarkers, (vi) informative censoring, and (vii) publication bias against quality-of-life data. Conclusion: Interpretation of the ESMO-MCBS scores requires critical appraisal of trials to understand caveats in trial design, implementation, and data analysis that may have biased results and conclusions. These will be addressed in future iterations of the ESMO-MCBS.

How aging of the global population is changing oncology

“…Dominance of clinical research by industry [ 43 ] has likewise selected for biases due to non-publication of negative studies [ 44 , 45 ], and trial designs favoring statistically significant outcomes [ 46 , 47 ] – even though many such outcomes provide little real-world benefit to patients [ 48 ], especially if older [ 49 ]. This is consistent with the impression that average trial gains have tended to reduce in recent decades [ 50 , 51 ], in part due to regulatory frameworks that have made therapeutic incrementalism the safest commercial strategy [ 52 ]. These caveats serve to remind us that clinical decisions should be made not by evidence alone, but by honest and self-critical discussion between doctors and patients [ 53 ], including older adults with cancer [ 54 ].…”

Pan-Asian adapted ESMO Clinical Practice Guidelines for the diagnosis, treatment and follow-up of patients with renal cell carcinoma

“… c More than 30% of control arm patients never received subsequent immunotherapy, suboptimal post progression treatment may exaggerate OS benefit. 88 d Calculated estimate of gain based on point estimate HR 0.68. e QoL evaluated as an exploratory endpoint (as distinct from primary or secondary endpoint) is not eligible for ESMO-MCBS grading.…”

“… c More than 30% of control arm patients never received subsequent immunotherapy, suboptimal post progression treatment may exaggerate OS benefit. 88 …”

Skip pattern approach toward the early access of innovative anticancer drugs

“…Study design, efficacy and activity endpoints as well as patients’ follow-up or patient-reported outcomes should be collected during trials to support the level of evidence, which should also be considered in the drug development and approval path. Finally, it should be considered, as published recently, 30 that methodological aspects related to trial design and statistical analysis can affect the trial’s interpretation and conclusions. Such methodological aspects should be taken into account during the appraisal process of innovative drugs.…”