Systematic bias between blinded independent central review and local assessment: literature review and analyses of 76 phase III randomised controlled trials in 45 688 patients with advanced solid tumour

Zhang, Jianrong; Zhang, Yiyin; Tang, Si; Jiang, Long; He, Qihua; Hamblin, Lindsey Tristine; He, Jiaxi; Xu, Zhiheng; Wu, Jing‐Ming; Chen, Yaoqi; Liang, Hengrui; Chen, Difei; Huang, Yu Chen; Wang, Xinyu; Deng, Kexin; Jiang, Shuhan; Zhou, Jun; Xu, Jiaxuan; Chen, Xuanzuo; Liang, Wenhua; He, Jianxing

doi:10.1136/bmjopen-2017-017240

Cited by 20 publications

(14 citation statements)

References 119 publications

(29 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similarly, independent clinicians could perform blinded central evaluation of tumour assessment endpoints, but this might have major cost implications 78. In a recent review of randomised controlled trials in solid tumours, there was no systematic bias between the findings from blinded independent central review and local assessment, but there were statistical inconsistencies between the two sets of results in almost a quarter of trials 79. Moreover, previous meta-epidemiological reviews across different therapeutic areas have found that studies with non-blinded assessors of subjective outcomes generate biased findings 4980.…”

Section: Discussionmentioning

confidence: 99%

Design characteristics, risk of bias, and reporting of randomised controlled trials supporting approvals of cancer drugs by European Medicines Agency, 2014-16: cross sectional analysis

Naci

Davis

Savović

et al. 2019

BMJ

129

100

View full text Add to dashboard Cite

ObjectiveTo examine the design characteristics, risk of bias, and reporting adequacy of pivotal randomised controlled trials of cancer drugs approved by the European Medicines Agency (EMA).DesignCross sectional analysis.SettingEuropean regulatory documents, clinical trial registry records, protocols, journal publications, and supplementary appendices.Eligibility criteriaPivotal randomised controlled trials of new cancer drugs approved by the EMA between 2014 and 2016.Main outcome measuresStudy design characteristics (randomisation, comparators, and endpoints); risk of bias using the revised Cochrane tool (bias arising from the randomisation process, deviations from intended interventions, missing outcome data, measurement of the outcome, and selection of the reported result); and reporting adequacy (completeness and consistency of information in trial protocols, publications, supplementary appendices, clinical trial registry records, and regulatory documents).ResultsBetween 2014 and 2016, the EMA approved 32 new cancer drugs on the basis of 54 pivotal studies. Of these, 41 (76%) were randomised controlled trials and 13 (24%) were either non-randomised studies or single arm studies. 39/41 randomised controlled trials had available publications and were included in our study. Only 10 randomised controlled trials (26%) measured overall survival as either a primary or coprimary endpoint, with the remaining trials evaluating surrogate measures such as progression free survival and response rates. Overall, 19 randomised controlled trials (49%) were judged to be at high risk of bias for their primary outcome. Concerns about missing outcome data (n=10) and measurement of the outcome (n=7) were the most common domains leading to high risk of bias judgments. Fewer randomised controlled trials that evaluated overall survival as the primary endpoint were at high risk of bias than those that evaluated surrogate efficacy endpoints (2/10 (20%) v 16/29 (55%), respectively). When information available in regulatory documents and the scientific literature was considered separately, overall risk of bias judgments differed for eight randomised controlled trials (21%), which reflects reporting inadequacies in both sources of information. Regulators identified additional deficits beyond the domains captured in risk of bias assessments for 10 drugs (31%). These deficits included magnitude of clinical benefit, inappropriate comparators, and non-preferred study endpoints, which were not disclosed as limitations in scientific publications.ConclusionsMost pivotal studies forming the basis of EMA approval of new cancer drugs between 2014 and 2016 were randomised controlled trials. However, almost half of these were judged to be at high risk of bias based on their design, conduct, or analysis, some of which might be unavoidable because of the complexity of cancer trials. Regulatory documents and the scientific literature had gaps in their reporting. Journal publications did not acknowledge the key limitations of the available evidence identified in regulatory documents.

show abstract

Section: Discussionmentioning

confidence: 99%

Design characteristics, risk of bias, and reporting of randomised controlled trials supporting approvals of cancer drugs by European Medicines Agency, 2014-16: cross sectional analysis

Naci

Davis

Savović

et al. 2019

BMJ

129

100

View full text Add to dashboard Cite

show abstract

“…In a large meta-analysis conducted over 76 phase 3 trials on a total of more than 45 000 patients with solid tumors, Zhang et al 31 calculated the statistical inconsistencies between the end points assessed by locals investigators vs the assessments by independent reviewers; a statistically inconsistent inference was defined as a condition whereby the treatment effect from one of the assessments shows a significant difference, while showing no significant difference from the other assessment. These authors found no significant differences in the treatment effects of ORR between blinded independent central review and local investigators (odds ratio, 1.02; 95%CI, 0.97-1.07), and similar findings were obtained with PFSs.…”

Section: Discussionmentioning

confidence: 99%

“…These authors found no significant differences in the treatment effects of ORR between blinded independent central review and local investigators (odds ratio, 1.02; 95%CI, 0.97-1.07), and similar findings were obtained with PFSs. 31 However, when these authors carried out a direct comparison of response rates between central and local assessments (data obtained from 28 phase 3 trials on solid tumors), they found significantly lower ORR values associated with blinded independent central review assessment compared to those of local investigators, for both experimental (odds ratio, 0.81; 95%CI, 0.76-0.87) and control arms (odds ratio, 0.79; 95%CI, 0.72-0.85). 32 Taken collectively, our data and those of Zhang et al 32 would suggest that the ORRs assessed by local investigators are significantly higher by about 20% compared to those assessed by blinded reviewers, regardless of whether the ORR assessment was carried out in phase 2 or in phase 3 trials.…”

Section: Discussionmentioning

confidence: 99%

Local Investigators Significantly Overestimate Overall Response Rates Compared to Blinded Independent Central Reviews in Phase 2 Oncology Trials

Russo

Cappoli

Pilunni

et al. 2020

The Journal of Clinical Pharma

View full text Add to dashboard Cite

The overall response rate (ORR) is a largely adopted outcome measure in early-phase oncology trials. ORR is highly relevant in cancer drug development at the time of deciding whether to move to confirmatory phase 3 trials; moreover, ORR is gaining increasing relevance in fast-track registration procedures. No systematic analysis has been conducted so far to investigate whether a discrepancy exists between ORR assessed by local investigators and those assessed by blinded reviewers in phase 2 oncology trials. In this study, we carried out a search in the clinicaltrials.gov and EudraCT databases, looking at the trials reporting the results of both investigator-assessed and independently-assessed ORR. A discrepancy index was obtained by calculating the ratio of each investigator-assessed ORR on the corresponding independently assessed ORR, so that a discrepancy index >1 indicates that the investigator was "more optimistic," whereas a discrepancy index <1 indicates the opposite. We also analyzed different subgroups (by tumor type, by drug type, by year). Twenty trials met the search criteria; in some cases, >1 comparison was conducted in the trial, so that the total number of comparisons analyzed was 33. The estimated mean discrepancy index was 1.175 (95% confidence interval, 1.083-1.264; n = 33). In conclusion, local investigators significantly overestimate ORR compared to paired blinded reviewers in phase 2 oncology trials. This may represent a risk in drug development, when deciding whether to move to confirmatory, more expensive phase 3 trials. Blinded independent central review should be used in ORR assessment if a more conservative estimate of treatment efficacy is required, as in the case of fast-track drug developments leading to accelerated approvals of cancer therapies.

show abstract

“…Independent reviews are conducted primarily to discern and minimise bias that may be introduced by the investigators (28). As such, blinded independent reviews are recommended for clinical trials studying tumour response or disease progression (28)(29)(30)(31)(32).…”

Section: Discussionmentioning

confidence: 99%

A randomised, multicentre open-label phase II study to evaluate the efficacy, tolerability and pharmacokinetics of oral vinorelbine plus cisplatin versus intravenous vinorelbine plus cisplatin in Chinese patients with chemotherapy-naive unresectable or metastatic non-small cell lung cancer

et al. 2019

View full text Add to dashboard Cite

Background: A phase II study to evaluate the efficacy, tolerability and pharmacokinetics of oral or intravenous vinorelbine (VRL) plus cisplatin (CDDP) in Chinese patients with non-small cell lung cancer (NSCLC). Methods: One hundred and thirty-one patients were randomised to oral VRL 60 mg/m 2 (arm A) or intravenous VRL 25 mg/m 2 (arm B) on days 1 and 8, plus CDDP 80 mg/m 2 on day 1 (both arms). VRL was increased to 80 mg/m 2 (arm A) or 30 mg/m 2 (arm B) in cycles 2-4 in the absence of toxicity. Primary efficacy endpoint was objective response rate (ORR). VRL pharmacokinetics was evaluated for possible drug-drug interactions with CDDP. Results: ORR was 25.8% in arm A and 23.1% in arm B. Disease control rate was 72.7% in arm A, 72.3% in arm B. Median overall survival was 16.1 months in arm A and 19.0 months in arm B. Median progressionfree survival was 4.6 months in arm A and 4.9 months in arm B. Forty-three point nine percent and 86.2% of patients had grade 3/4 neutropenia in arms A and B, respectively; incidence of febrile neutropenia was low (6.1% and 9.2%, respectively). Frequency of grade 3/4 non-haematological adverse events was also low. VRL pharmacokinetics was not affected by co-administration of CDDP.Conclusions: Oral and intravenous VRL in combination with CDDP is effective and well-tolerated in Chinese patients with advanced NSCLC. VRL pharmacokinetics is unaffected by CDDP co-administration.Oral VRL could be an effective alternative to intravenous VRL as a first-line treatment for NSCLC, as it optimises treatment convenience while maintaining high efficacy.

show abstract

Systematic bias between blinded independent central review and local assessment: literature review and analyses of 76 phase III randomised controlled trials in 45 688 patients with advanced solid tumour

Cited by 20 publications

References 119 publications

Design characteristics, risk of bias, and reporting of randomised controlled trials supporting approvals of cancer drugs by European Medicines Agency, 2014-16: cross sectional analysis

Design characteristics, risk of bias, and reporting of randomised controlled trials supporting approvals of cancer drugs by European Medicines Agency, 2014-16: cross sectional analysis

Local Investigators Significantly Overestimate Overall Response Rates Compared to Blinded Independent Central Reviews in Phase 2 Oncology Trials

A randomised, multicentre open-label phase II study to evaluate the efficacy, tolerability and pharmacokinetics of oral vinorelbine plus cisplatin versus intravenous vinorelbine plus cisplatin in Chinese patients with chemotherapy-naive unresectable or metastatic non-small cell lung cancer

Contact Info

Product

Resources

About