Design characteristics, risk of bias, and reporting of randomised controlled trials supporting approvals of cancer drugs by European Medicines Agency, 2014-16: cross sectional analysis

Abstract

ObjectiveTo examine the design characteristics, risk of bias, and reporting adequacy of pivotal randomised controlled trials of cancer drugs approved by the European Medicines Agency (EMA).DesignCross sectional analysis.SettingEuropean regulatory documents, clinical trial registry records, protocols, journal publications, and supplementary appendices.Eligibility criteriaPivotal randomised controlled trials of new cancer drugs approved by the EMA between 2014 and 2016.Main outcome measuresStudy design characteristics (randomisation, comparators, and endpoints); risk of bias using the revised Cochrane tool (bias arising from the randomisation process, deviations from intended interventions, missing outcome data, measurement of the outcome, and selection of the reported result); and reporting adequacy (completeness and consistency of information in trial protocols, publications, supplementary appendices, clinical trial registry records, and regulatory documents).ResultsBetween 2014 and 2016, the EMA approved 32 new cancer drugs on the basis of 54 pivotal studies. Of these, 41 (76%) were randomised controlled trials and 13 (24%) were either non-randomised studies or single arm studies. 39/41 randomised controlled trials had available publications and were included in our study. Only 10 randomised controlled trials (26%) measured overall survival as either a primary or coprimary endpoint, with the remaining trials evaluating surrogate measures such as progression free survival and response rates. Overall, 19 randomised controlled trials (49%) were judged to be at high risk of bias for their primary outcome. Concerns about missing outcome data (n=10) and measurement of the outcome (n=7) were the most common domains leading to high risk of bias judgments. Fewer randomised controlled trials that evaluated overall survival as the primary endpoint were at high risk of bias than those that evaluated surrogate efficacy endpoints (2/10 (20%) v 16/29 (55%), respectively). When information available in regulatory documents and the scientific literature was considered separately, overall risk of bias judgments differed for eight randomised controlled trials (21%), which reflects reporting inadequacies in both sources of information. Regulators identified additional deficits beyond the domains captured in risk of bias assessments for 10 drugs (31%). These deficits included magnitude of clinical benefit, inappropriate comparators, and non-preferred study endpoints, which were not disclosed as limitations in scientific publications.ConclusionsMost pivotal studies forming the basis of EMA approval of new cancer drugs between 2014 and 2016 were randomised controlled trials. However, almost half of these were judged to be at high risk of bias based on their design, conduct, or analysis, some of which might be unavoidable because of the complexity of cancer trials. Regulatory documents and the scientific literature had gaps in their reporting. Journal publications did not acknowledge the key limitations of the available evidence identified in regulatory documents.

Views of European Drug Development Stakeholders on Treatment Optimization and Its Potential for Use in Decision-Making

“…An extensive analysis of the FDA's oncological medicine approvals between 2006 and 2015 affirmed these observations (Rodríguez et al, 2019). Furthermore, clinical trials providing the evidence needed to underpin the regulatory approval process (hereinafter referred to as "registrational trials") can be prone to bias, which often remains inadequately reported (Naci et al, 2019). Moreover, the accelerated and conditional marketing authorization mechanisms launched by FDA and EMA respectively have further solidified the use of surrogate endpoints in clinical trials (Fleming, 2005;Downing et al, 2014;Hoekman et al, 2015).…”

FDA and EMA clinical research guidelines: Assessment of trial design recommendations for pivotal psychiatric drug trials (Protocol)

“…Uncertain patient-relevant benefits of new drugs Several recent reports have found that newly authorised medicines often have questionable or no added patient-relevant benefits compared to already available, older treatments. [1][2][3][4][5][6] In a cohort of 216 drugs approved by the German drug regulatory authority between 2011 and 2016, the independent Institute for Quality and Efficiency in Health Care (IQWiG) judged that 22 (10 %) new drugs had substantial benefits compared to already available treatments, while 125 (58 %) drugs had no proof of added benefits. 1 Cohort studies of oncology drugs approved by the European Medicines Agency (EMA) 4 and the US Food and Drug Administration (FDA) 5 found that most drugs were approved without evidence of benefits on overall mortality or quality of life.…”

“…Similarly, an analysis of oncology drugs approved by the EMA between 2014 and 2016 reported that many pivotal trials underpinning drug approvals were of high risk of bias due to problems with the trial design and or trial conduct. 6 It has been argued that the threshold for new drug approvals has been lowered to accommodate the pharmaceutical industry's interests, 7 and researchers have advocated for more pragmatic, patient-relevant trials. 8 Field specific issues in psychiatry Systematic reviews, particularly of antidepressants for depression 9, 10 and of central stimulants for attention deficit hyperactivity disorder, [11][12][13] have highlighted problems of low generalizability of psychiatric drug trials, similar to those identified in pivotal oncology drug trials.…”

Concordance as evidence in the Watson for Oncology decision-support system

“…Ledford (2019), for example, argues that ethnicity is a major concern for geneticists with regard to studying cancer across populations, but is often overlooked in Western clinical trials for cancer. A recent study on cancer drugs approved by the European Medicines Agency (EMA) argued that almost all studies between 2014 and 2016 were at risk of bias (Naci et al 2019). Others have suggested that trials conducted by the pharmaceutical industry, as opposed to academic investigators alone, differ vastly with regard to their goals, whereby studies conducted by the pharmaceutical industry may not have the best interests of the patient as a concern, but rather focus on drug approval (Piccart et al 2007).…”