False Discovery Rate Estimation for Frequentist Pharmacovigilance Signal Detection Methods

Abstract

Pharmacovigilance systems aim at early detection of adverse effects of marketed drugs. They maintain large spontaneous reporting databases for which several automatic signaling methods have been developed. One limit of those methods is that the decision rules for the signal generation are based on arbitrary thresholds. In this article, we propose a new signal-generation procedure. The decision criterion is formulated in terms of a critical region for the P-values resulting from the reporting odds ratio method as well as from the Fisher's exact test. For the latter, we also study the use of mid-P-values. The critical region is defined by the false discovery rate, which can be estimated by adapting the P-values mixture model based procedures to one-sided tests. The methodology is mainly illustrated with the location-based estimator procedure. It is studied through a large simulation study and applied to the French pharmacovigilance database.

Data mining methodologies for pharmacovigilance

“…More recently, Ahmed et al [48,49] proposed false discovery rate (FDR) estimation for the frequentist methods to address the limitation of arbitrary thresholds. As of now, there is no consensus on which DPA method is better because there is no gold standard dataset available to evaluate the performances of the methods.…”

Quantitative Signal Detection and Analysis in Pharmacovigilance

“…The limitations discussed in the previous section are well understood and their implications appreciated; so, given that no method is perfect, there is a relentless pursuit to produce ever more effective methods. Efforts include use of different priors in Bayesian false discovery rates (Gould, 2003) and false discovery rates in a frequentist context (Ahmed et al, 2010); and many other measures have been proposed, such Many biases can affect the results of DA methods used for signal detection in spontaneous reporting databases, because of the very nature of spontaneous reporting. The results of these analyses, taken in isolation, should not be overinterpreted as stronger evidence than they are: they remain statistical associations based on occurrence of reports, and can only be translated into safety signals once the above biases have been eliminated and the cases they rely on have been clinically assessed by pharmacovigilance experts.…”

Modern Issues and Methods in Biostatistics

“…They maintain large spontaneous reporting databases (SRDs) for which several automatic signaling methods have been developed (Ahmed et al 2010). The commonly used public safety databases include FDA Spontaneous Report System (SRS) (postmarketing surveillance of all drugs since 1969), FDA Adverse Event Reporting System (AERS), FDA/CDC Vaccine Adverse Events (VAERS), World Health Organization Collects Similar Data across Countries, and others.…”