Evaluation of quantitative signal detection in EudraVigilance for orphan drugs: possible risk of false negatives

Abstract

Different strategies have been studied to allow a better characterization of the safety profile of orphan drugs soon after their approval. At the end of the development phases only few data are available because of the small number of subjects exposed to an orphan medicine for the treatment of rare or ultra-rare conditions. As a consequence, the evaluation of the safety profile is limited at the time of the first approval. In the post-marketing period, all available sources should be combined for a better understanding of the safety of orphan drugs. These sources, include outputs from large databases such as the European Medicines Agency’s EudraVigilance database. Analyses of data from this source are required to be performed by marketing authorization holders (MAHs) as part of their signal management activities. In 2018, the Pharmacovigilance Risk Assessment Committee (PRAC) assessed 114 confirmed signals, 79% of which included data from EudraVigilance. MAHs have access to statistical calculations for drug–event combinations (DECs) from EudraVigilance, provided in the form of measures of disproportionality of ratios of the observed proportion of spontaneous cases for a DEC in relation to the proportion of cases that would be expected if no association existed between the drug and the event. However, such statistical summaries for orphan drugs could be misleading because of the very limited safety data available for orphan drugs (under-reporting together with low numbers of exposed patients). In addition, the applied statistical methodology in most instances is constrained by different confounding factors such as indications of specific medicines and the wide spectrum of medical conditions/diseases of patients from whom reporting of disproportionality ratios are derived (i.e. proportions of DECs for orphan drugs (ODECs) from a small patient population suffering the rare disease and the proportion of DECs in the rest of the population represented in the whole database who have been treated with other medicines for a wide range of indications, and prescribed to treat completely different medical conditions). As expected, these statistical calculations produced not only signals of disproportionate reporting (SDRs) that are false positives, but also not sensitive enough to detect certain SDRs, thus resulting in false negatives. In the context of rare/ultra-rare life-threatening diseases where new molecules have been made available on the market on the basis of their proven efficacy, but with only limited safety data at the time of approval, false negatives could be a special concern since unlikely converted in positives or becoming positives with notable delay. Subgroup analyses (using a limited dataset comprising ADRs within specific individual case safety reports (ICSRs), sorted by indication/disease relevant to the drug of interest could, at least in part, possibly reduce some of the weaknesses resulting from the abovementioned confounding factors. On the other hand it could also cause the loss of some identification of SDRs that would be captured if no database restrictions had been undertaken. Therefore, data subgroup analysis should not be selected as a preferred approach to quantitative signal detection for orphan drugs but rather evaluated as complementary possibly to confirm negatives or to further characterize detected SDRs. Some examples of false negatives originating from quantitative signal detection in EudraVigilance applied to orphan drugs are discussed in this article.

Analysis of Thrombotic Adverse Reactions of COVID-19 AstraZeneca Vaccine reported to EudraVigilance database

“…Their natural incidence ranges from 56 events per 100,000 persons to 182 events per 100,000 persons which should be considered against the three deaths from PE in 17 million following vaccination [16,17]. A recent single study from Croatia reported an increase in prevalence of 'combined DVT with PE but not with isolated PE or isolated DVT' [18]. The authors noted from their comparison of non-Covid-19 patients in matched 8-month periods in 2019 pre-and in 2020 during the pandemic a significantly older age group admitted to hospital (60.8 ± 17.2 years v. 68.5± 16.8 years) [18].…”

Machine Learning for Detection of Safety Signals From Spontaneous Reporting System Data: Example of Nivolumab and Docetaxel

“…Therefore, it is important to detect rare AEs through post-marketing surveillance studies. However, traditional methods are not sensitive enough to detect rare AEs, resulting in false negatives ( Sardella and Lungu, 2019 ). Hence, several regulatory authorities recommend that DPAs be complemented with other appropriate methods to identify rare safety signals ( Kimura et al, 2011 ; Jos, 2015 ; Hesha, 2018 ).…”

European ad-hoc consensus statement on gene replacement therapy for spinal muscular atrophy

“…All contributing parties should make these data available in an unbiased manner. Identification of new safety signals is specifically important for orphan drugs where the experience is still limited at the time of approval [35]. Therefore, any new safety signal should not only be communicated with clinical trial sites but with all treatment centres.…”