Multi‐centre, multi‐database studies with common protocols: lessons learnt from the IMI PROTECT project

Abstract

Purpose To assess the impact of a variety of methodological parameters on the association between six drug classes and five key adverse events in multiple databases. Methods The selection of Drug-Adverse Event pairs was based on public health impact, regulatory relevance, and the possibility to study a broad range of methodological issues. Common protocols and data analytical specifications were jointly developed and independently and blindly executed in different databases in Europe with replications in the same and different databases. Results The association between antibiotics and acute liver injury, benzodiazepines and hip fracture, antidepressants and hip fracture, inhaled long-acting beta2-agonists and acute myocardial infarction was consistent in direction across multiple designs, databases and methods to control for confounding. Some variation in magnitude of the associations was observed depending on design, exposure and outcome definitions, but none of the differences were statistically significant. The association between anti-epileptics and suicidality was inconsistent across the UK CPRD, Danish National registries and the French PGRx system. Calcium channel blockers were not associated with the risk of cancer in the UK CPRD, and this was consistent across different classes of calcium channel blockers, cumulative durations of use up to >10 years and different types of cancer. Conclusions A network for observational drug effect studies allowing the execution of common protocols in multiple databases was created. Increased consistency of findings across multiple designs and databases in different countries will increase confidence in findings from observational drug research and benefit/risk assessment of medicines.

Nonrandomized Real‐World Evidence to Support Regulatory Decision Making: Process for a Randomized Trial Replication Project

“…Although we have made every effort to set up a RWE study implementation process that prevents study outcomes and results from influencing the conduct of the study, it is still possible that knowledge of the RCT finding that is being targeted may influence the study because investigators are not blinded to the RCT results. One way to avoid this limitation would be to blind investigators to trial results until completion of the study, similar to the approach taken by the Innovative Medicines Initiative‐Pharmacoepidemiological Research on Outcomes of Therapeutics by a European Consortium (IMI‐PROTECT) initiative, although it would be impossible to guarantee that investigators did not learn of trial findings outside of the project . Therefore, a second project has been launched to attempt to predict the results of a set of ongoing RCTs, where results have not yet been released, using very similar methodology.…”

“…One way to avoid this limitation would be to blind investigators to trial results until completion of the study, similar to the approach taken by the Innovative Medicines Initiative-Pharmacoepidemiological Research on Outcomes of Therapeutics by a European Consortium (IMI-PROTECT) initiative, although it would be impossible to guarantee that investigators did not learn of trial findings outside of the project. 39,40 Therefore, a second project has been launched to attempt to predict the results of a set of ongoing RCTs, where results have not yet been released, using very similar methodology.…”

Lessons from meta‐analyses of randomized clinical trials for analysis of distributed networks of observational databases

“…This may be less of a concern in a network of relatively homogenous transactional insurance claims. In other observational networks with more varied data sources, eg, those in Europe across both EMRs and insurance claims, this additional information could be very helpful …”

“Adaptive pathways” to drug authorisation: adapting to industry?

“…2 Yet past experience has shown that different observational studies investigating the same safety issue may produce conflicting results, 24 and recent evaluations of regulators' and researchers' experience of using large, electronic health databases indicate mixed results with respect to signal detection and signal confirmation. [25][26][27][28] One review found that large, publicly funded pilot studies in the EU and US have largely failed to provide credible evidence of new, unsuspected adverse effects or to yield reproducible results. The review authors argue that considerable progress is needed in database terminology and coding, data validation, and statistical methods before electronic health records can be a reliable resource for rapid assessment of drug safety.…”