Benchmarking single-arm studies against historical controls from non-small cell lung cancer trials – an empirical analysis of bias

Abstract

Background: Recent trials of novel agents in 'rare' molecular subtypes of non-small cell lung cancer (NSCLC) have used single-arm trial designs and benchmarked outcomes against historical controls. We assessed the consistency of historical control outcomes using docetaxel data from published NSCLC randomized controlled trials (RCTs). Material and methods: Advanced NSCLC RCTs including a docetaxel monotherapy arm were included. Heterogeneity in tumor objective response rates (ORRs), progression-free survival (PFS) and overall survival (OS), and correlations between outcomes and year of trial commencement were assessed. Results: Among 63 trials (N ¼ 10,633) conducted between 2000 and 2017, ORR ranged from 0% to 26% (I 2 ¼ 76.1%, p heterogeneity < .0001). Mean of the median PFS was 3.0 months (range: 1.4-6.4), 3month PFS ranged from 25% to 85% (I 2 ¼ 86.0%, p heterogeneity < .0001). Mean of the median OS was 9.1 months (range: 4.7-22.9), 9-month OS ranged from 23% to 79% (I 2 ¼ 83.0%, p heterogeneity < .0001). Each later year of trial commencement was associated with 0.3% (p ¼ .046), 0.5% (p ¼ .11) and 0.9% (p ¼ .001) improvement in ORR, 3-month PFS and 9-month OS rates, respectively. Conclusions: There was significant heterogeneity and an improving trend in docetaxel outcomes across trials conducted over 20 years. Benchmarking biomarker-targeted agents against historical controls may not be a valid approach to replace RCTs. Innovative study designs involving a concurrent control arm should be considered.

Summarising salient information on historical controls: A structured assessment of validity and comparability across studies

“…27 Drift due to improvements in care over time is also an important concern in practice. In a meta-analysis by Snyders et al, 33 who looked at 63 trials of docetaxel in lung cancer (enrolling over 10,000 patients), it was found that outcomes improved each year by a mean of 0.3% in objective response rate (ORR), 0.5% for progression-free survival (PFS), and 0.9% in OS. These changes over time imply that historical data that are more than say 10 years old for docetaxel in this indication should be avoided.…”

Framework for the Use of External Controls to Evaluate Treatment Outcomes in Precision Oncology Trials

“…Comparative effectiveness analyses incorporating ECs are at risk of inherent biases associated with observational data. [52][53][54][55][56][57][58][59][60] Common types of biases include selection bias, information bias, and time-related bias (Table 1). These biases should be considered early when the study is conceptualized and comprehensively addressed in the SAP.…”

Surrogate endpoints for HTA decisions of breast cancer drugs: utility and pitfalls

“…An additional issue with ORR based drug approval is the lack of a control arm since these are usually from single arm trials. In the absence of a control, comparative analysis and cost-effectiveness analysis become impossible to inform HTA decisions and HTA agencies will have to rely on ORR from historical control which is fraught with several biases [18,19]. Even then, the effect on survival remains unknown.…”