Ten Rules for Conducting Retrospective Pharmacoepidemiological Analyses: Example COVID-19 Study

Abstract

Since the beginning of the COVID-19 pandemic, pharmaceutical treatment hypotheses have abounded, each requiring careful evaluation. A randomized controlled trial generally provides the most credible evaluation of a treatment, but the efficiency and effectiveness of the trial depend on the existing evidence supporting the treatment. The researcher must therefore compile a body of evidence justifying the use of time and resources to further investigate a treatment hypothesis in a trial. An observational study can provide this evidence, but the lack of randomized exposure and the researcher’s inability to control treatment administration and data collection introduce significant challenges. A proper analysis of observational health care data thus requires contributions from experts in a diverse set of topics ranging from epidemiology and causal analysis to relevant medical specialties and data sources. Here we summarize these contributions as 10 rules that serve as an end-to-end introduction to retrospective pharmacoepidemiological analyses of observational health care data using a running example of a hypothetical COVID-19 study. A detailed supplement presents a practical how-to guide for following each rule. When carefully designed and properly executed, a retrospective pharmacoepidemiological analysis framed around these rules will inform the decisions of whether and how to investigate a treatment hypothesis in a randomized controlled trial. This work has important implications for any future pandemic by prescribing what we can and should do while the world waits for global vaccine distribution.

Challenges in evaluating treatments for COVID-19: The case of in-hospital anticoagulant use and the risk of adverse outcomes

“…Several initiatives have been launched to provide timely access to the large databases with multiple data sources, such as the International 4CE Consortium ( Brat et al, 2020 ), National COVID Cohort Collaborative (N3C) ( Bennett et al, 2021 ) in the U.S., and the database used in our study. With the appropriate data sources in place, investigators from multiple disciplines, and rigorous study design ( Franklin et al, 2021 ; Powell et al, 2021 ), observational studies can be useful to help inform the constantly evolving evidence on treatments for COVID-19.…”

Federated causal inference in heterogeneous observational data

“…i ) estimated using the estimated outcome and propensity models on data set j. ‡ ‡ ‡ Our analysis builds on the studies by Konig et al, 49 Koenecke et al, 1 Rose et al, 50 and Powell et al 51 § § § Note that our findings reproduce similar results to Koenecke et al, 1 validating the prior result suggesting that alpha blockers are effective in reducing ventilation and death in ARD and pneumonia patients; however, the confidence levels are narrower because our federated methods presented here are improved from those used in Koenecke et al 1 Koenecke et al 1 only use the treatment coefficient and variance in federation, whereas here, we use the full variance-covariance matrix from all covariates. Our approach leverages the stable part of the model across two data sets, which could improve the estimation precision of the treatment coefficient.…”

“…‡‡‡‡‡‡ Our analysis builds on the studies by Konig et al, 49 Koenecke et al, 1 Rose et al, 50 and Powell et al 51 …”

When no answer is better than a wrong answer: a causal perspective on batch effects

“…As there are likely many other categories with which brain connectivity may be confounded, we do not believe this covariate set is sufficient to identify a Causal Batch Effect 1, as we would need to be confident that these covariates exhibited the covariate sufficiency property. We perform exact matching on the basis of individual sex and individual continent of measurement, and use a 0.1-standard deviation caliper on the propensity score to obtain at most k matched participants for each treated individual [35].…”