Learning From Epidemiology: Interpreting Observational Database Studies for the Effects of Medical Products

Abstract

Causal assessment of adverse effects continues to evolve through the medical product lifecycle and requires clinical judgment to integrate evidence from multiple sources, including observational studies. This study describes a probabilistic framework quantifying how much can be learned from an epidemiological study, and how it varies by study design, database, or prior beliefs. We integrate new observational evidence with existing clinical knowledge by estimating the probability that a drug-outcome pair represents a true association, as a function of prior belief and estimated strength of association from an observational study. Incident user cohort with propensity score adjustment and self-controlled case series designs were applied in an automated fashion to 53 drug-outcome associations across a network of nine disparate databases covering over 100 million lives of patient-level data. When applying a incident user design to a large claims database for a drug with moderate sample, the performance of these method suggests that an observed relative risk of less than 2 does not substantially shift prior expectations and that a relative risk of greater than 5 is needed to revise a 10% prior belief in a true association to have a greater than 90% posterior probability. Study design, database, and sample size can substantially impact the predictive performance of observational results. This Bayesian framework offers a conceptual approach to interpreting observational database studies, which, with further data to fit the model, offers the potential to become a practical tool for direct use by clinicians and the broader research commnunity.

Comparative evaluation of methods approximating drug prescription durations in claims data: modeling, simulation, and application to real data

“…Thus, more refined future approaches could also focus on temporarily varying COV approximations (e.g., similar to the extension of the proportion of days covered) or on more specific ways than the ’15‐day rule’ to address poor adherence while considering individual drugs and their specific administration regimens. Finally, unmeasured and possibly time‐dependent confounding cannot be ruled out with respect to factors influencing outcome, exposure, or being equally related to both (e.g., indication, contraindication, disease severity, or over‐the‐counter drugs) …”

“…Finally, unmeasured and possibly timedependent confounding cannot be ruled out with respect to factors influencing outcome, exposure, or being equally related to both (e.g., indication, contraindication, disease severity, or over-the-counter drugs). [46][47][48][49]…”

Signal Detection for Recently Approved Products: Adapting and Evaluating Self-Controlled Case Series Method Using a US Claims and UK Electronic Medical Records Database

“…We note that five pairs (Herpes Zoster, Lymphoma, and Pneumonia with Adalimumab; Orthostatic Hypotension with desvenlafaxine; and Orthostatic Hypotension with escitalopram in Optum) were all highlighted by the "Exposure Duration" approach with an IRR≥2. Some argue that observational studies are more reliable when estimates greater than 2 are achieved [37][38][39], and in signal detection ranking based on SDR is often used to prioritise of pairs for further consideration [5].…”

Integrating database knowledge and epidemiological design to improve the implementation of data mining methods that evaluate vaccine safety in large healthcare databases

“…However, there are many obstacles to overcome when adapting data mining methods for use in this setting. General challenges of using observational healthcare data to evaluate the effects of medical products have recently been described [17–19]. In this study, we discuss specific issues encountered when utilizing such data sources in data mining applications to evaluate vaccine safety.…”