Improved Statistical Signal Detection in Pharmacovigilance by Combining Multiple Strength-of-Evidence Aspects in vigiRank

Abstract

BackgroundDetection of unknown risks with marketed medicines is key to securing the optimal care of individual patients and to reducing the societal burden from adverse drug reactions. Large collections of individual case reports remain the primary source of information and require effective analytics to guide clinical assessors towards likely drug safety signals. Disproportionality analysis is based solely on aggregate numbers of reports and naively disregards report quality and content. However, these latter features are the very fundament of the ensuing clinical assessment.ObjectiveOur objective was to develop and evaluate a data-driven screening algorithm for emerging drug safety signals that accounts for report quality and content.MethodsvigiRank is a predictive model for emerging safety signals, here implemented with shrinkage logistic regression to identify predictive variables and estimate their respective contributions. The variables considered for inclusion capture different aspects of strength of evidence, including quality and clinical content of individual reports, as well as trends in time and geographic spread. A reference set of 264 positive controls (historical safety signals from 2003 to 2007) and 5,280 negative controls (pairs of drugs and adverse events not listed in the Summary of Product Characteristics of that drug in 2012) was used for model fitting and evaluation; the latter used fivefold cross-validation to protect against over-fitting. All analyses were performed on a reconstructed version of VigiBase® as of 31 December 2004, at around which time most safety signals in our reference set were emerging.ResultsThe following aspects of strength of evidence were selected for inclusion into vigiRank: the numbers of informative and recent reports, respectively; disproportional reporting; the number of reports with free-text descriptions of the case; and the geographic spread of reporting. vigiRank offered a statistically significant improvement in area under the receiver operating characteristics curve (AUC) over screening based on the Information Component (IC) and raw numbers of reports, respectively (0.775 vs. 0.736 and 0.707, cross-validated).ConclusionsAccounting for multiple aspects of strength of evidence has clear conceptual and empirical advantages over disproportionality analysis. vigiRank is a first-of-its-kind predictive model to factor in report quality and content in first-pass screening to better meet tomorrow’s post-marketing drug safety surveillance needs.

Patient-Reported Safety Information: A Renaissance of Pharmacovigilance?

“…In most cases of statistical signal detection, a measure of disproportionality is calculated for every report, despite the (quality of) information it contains. New approaches to statistical signal detection try to take the (quality of) the information into account [33]. However, a limitation of statistical signal detection is that it only takes structured fields into account, not making use of data in unstructured fields.…”

What Is the Plural of a ‘Yellow’ Anecdote?

“…The most extreme PRRs were cases 5, 6 and 8 in FEDRA, illustrating that the variance of PRR with small numbers is very high. This suggests that one of the 'shrinkage' or Bayesian methods may be more useful; the adaptation of the 'information component' (IC) from the Uppsala group is simple and, combined with the addition of other information in their 'vigiRank' system, may have considerable utility [12]. Whether this will help when the total number of reports is smaller than, say, 100 is not yet known.…”

Quantitative Methods for the Identification of Signals for Individual Case Safety Reports in India

“…In literature published by the WHO-UMC, it has been reported that the IC (lower bound [IC 025 ] > 0) is a criterion for signal identification. 9,11 However, practical experience as well as formal validation of the test cases of drug-ADR pairs and expert comment on threshold values for these parameters were also used to identify true signals. The threshold values used in the PvPI for the aforementioned parameters to identify any potential signal were the following:…”