Larger effect sizes in nonrandomized studies are associated with higher rates of EMA licensing approval

Abstract

Nonrandomized data were used for 7% of EMA drug approvals. Larger effect sizes were associated with greater likelihood of approval based on nonrandomized data alone. We did not find a clear treatment effect threshold for drug approval without RCT evidence.

US Food and Drug Administration Approvals of Drugs and Devices Based on Nonrandomized Clinical Trials

“…Insufficient data were available to test whether an OR was 12 or greater, which we had identified as an empirical value for dramatic effects in the EMA study. 10 Statistical analyses were conducted in R version 3.…”

The importance of randomised vs non-randomised trials

“…3 Ideally, health policy changes (eg, introducing or stopping vaccines) should be done in a manner that allows for proper assessment of their effect on overall health. For instance, concerns about the negative sideeffects of the human papillomavirus vaccine could have been addressed more appropriately if the first cohorts had been randomly allocated to Figure: Preference for further RCTs by the EMA and the US FDA according to three threshold criteria 1-3 EMA data (71 studies) 4 and US FDA data (36 studies in the Breakthrough Therapy Designation programme, unpublished) are combined. Borderline or significant differences in the rates of requests for further RCTs were found when using an effect size threshold of RR >4 (p=0 decisions that are made in everyday clinical practice, these decisions, by necessity, have to be made on the basis of the best available evidence from observational studies.…”

“…This internationally notable portfolio includes large, phase 3, randomised controlled trials, in which study drugs are often provided free of charge by pharmaceutical companies. 4 In addition to improving clinical outcomes, accelerated trial delivery drives economic growth and represents a core feature of the UK life sciences proposition, as articulated in HM Government's Life Sciences Industrial Strategy. 5 Blood cancer is a significant area of therapeutic advance.…”

“…However, we did not find evidence of a threshold estimate of effect beyond which no further randomised evidence would be required for licensing. 4 Application of the effect size threshold proposed by Gerstein and colleagues 1 (RR ≥4), or of the thresholds of RR ≥5 and RR ≥10, is associated with about a two-times higher rate of licensing approval without requiring further testing in RCTs (figure).…”

A review of NICE appraisals of pharmaceuticals 2000–2016 found variation in establishing comparative clinical effectiveness

“…In such cases, some observers have postulated that RCTs may not be necessary. (6)(7)(8) Additionally, in the context of small populations, RCTs may be unethical and misleading, as results could be statistically underpowered. (9) As a result, drug licensing agencies such as the European Medicines Agency (EMA) may grant marketing authorisations to pharmaceutical technologies with no RCT data, when there is certainty that the product's benefit outweighs potential harm (10).…”