Data Mining for Adverse Drug Events With a Propensity Score-matched Tree-based Scan Statistic

Abstract

The tree-based scan statistic is a statistical data mining tool that has been used for signal detection with a self-controlled design in vaccine safety studies. This disproportionality statistic adjusts for multiple testing in evaluation of thousands of potential adverse events. However, many drug safety questions are not well suited for self-controlled analysis. We propose a method that combines tree-based scan statistics with propensity score-matched analysis of new initiator cohorts, a robust design for investigations of drug safety. We conducted plasmode simulations to evaluate performance. In multiple realistic scenarios, tree-based scan statistics in cohorts that were propensity score matched to adjust for confounding outperformed tree-based scan statistics in unmatched cohorts. In scenarios where confounding moved point estimates away from the null, adjusted analyses recovered the prespecified type 1 error while unadjusted analyses inflated type 1 error. In scenarios where confounding moved point estimates toward the null, adjusted analyses preserved power, whereas unadjusted analyses greatly reduced power. Although complete adjustment of true confounders had the best performance, matching on a moderately mis-specified propensity score substantially improved type 1 error and power compared with no adjustment. When there was true elevation in risk of an adverse event, there were often co-occurring signals for clinically related concepts. TreeScan with propensity score matching shows promise as a method for screening and prioritization of potential adverse events. It should be followed by clinical review and safety studies specifically designed to quantify the magnitude of effect, with confounding control targeted to the outcome of interest.

A novel data mining application to detect safety signals for newly approved medications in routine care of patients with diabetes

“…Propensity score (PS) matching methodology was used to adjust for confounding using a nearest neighbour matching within a caliper of 0.05 17 . The probability of initiating canagliflozin versus a DPP4 inhibitor or a GLP1 agonist was calculated through a multivariable logistic regression model which contained all of the potential confounders at baseline.…”

“…TreeScan has also been used to determine if it can identify well‐established side effects of widely used medications, including diabetes medications and antifungal medications that have been in use for decades 10 . However, whether a more recently proposed method that combines TreeScan with propensity score‐matched analysis in the context of a new‐user active comparator study design can be used to reliably identify drug‐related adverse events among patients with diabetes using newly approved medications remains unknown 17 . Thus, we sought to evaluate whether TreeScan combined with propensity score matching and a new‐user active comparator design could help identify incident adverse events of a newly approved diabetes medication shortly after its market entry.…”

Data Mining for Adverse Events of Tumor Necrosis Factor-Alpha Inhibitors in Pediatric Patients: Tree-Based Scan Statistic Analyses of Danish Nationwide Health Data

“…To identify cuts with a higher incidence we used tree-based scan statistics, which are disproportionality statistics that adjust for multiple testing and that allow for simultaneous testing of diagnosis codes at all levels of granularity, that is, all cuts on the ICD-10 tree [ 15 ]. We screened for potential adverse events in the propensity score-matched analysis using the unconditional Bernoulli model [ 16 , 17 ]. Exposure was assumed to follow a Bernoulli probability distribution.…”

Real‐World Data, Advanced Analytics, and the Evolution of Postmarket Drug Safety Surveillance

“…While Sentinel has been used to date to study or refine specific drug safety signals generated from other sources, the FDA is now initiating pilot projects to examine the feasibility of this system to generate hypothesis-free signals. These projects will use a variety of approaches that can include advanced methods to control for confounding, including prospective sequential surveillance of select, pre specified outcomes using Sentinel's prospective sequential surveillance tool, surveillance of all outcomes associated with one particular product using the TreeScan methodology, 5 and surveillance of a particular outcome associated with any product using the DrugScan methodology. 6 As with other Sentinel capabilities, these approaches can be further refined as the FDA integrates them into its routine pharmacovigilance activities and gains experience with them.…”