Six Years of the US Food and Drug Administration's Postmarket Active Risk Identification and Analysis System in the Sentinel Initiative: Implications for Real World Evidence Generation

Abstract

Congress mandated the creation of a postmarket Active Risk Identification and Analysis (ARIA) system containing data on 100 million individuals for monitoring risks associated with drug and biologic products using data from disparate sources to complement the US Food and Drug Administration's (FDA's) existing postmarket capabilities. We report on the first 6 years of ARIA utilization in the Sentinel System (2016–2021). The FDA has used the ARIA system to evaluate 133 safety concerns; 54 of these evaluations have closed with regulatory determinations, whereas the rest remain in progress. If the ARIA system and the FDA's Adverse Event Reporting System are deemed insufficient to address a safety concern, then the FDA may issue a postmarket requirement to a product's manufacturer. One hundred ninety‐seven ARIA insufficiency determinations have been made. The most common situation for which ARIA was found to be insufficient is the evaluation of adverse pregnancy and fetal outcomes following in utero drug exposure, followed by neoplasms and death. ARIA was most likely to be sufficient for thromboembolic events, which have high positive predictive value in claims data alone and do not require supplemental clinical data. The lessons learned from this experience illustrate the continued challenges using administrative claims data, especially to define novel clinical outcomes. This analysis can help to identify where more granular clinical data are needed to fill gaps to improve the use of real‐world data for drug safety analyses and provide insights into what is needed to efficiently generate high‐quality real‐world evidence for efficacy.

Transparency, reproducibility, and replicability of pharmacoepidemiology studies in a distributed network environment

“…Another limitation that is common to all predominantly health insurance claims‐based databases is the inability to identify several medical conditions of interest to a satisfactory level of accuracy. Inability to identify the health outcome of interest was one of the reasons for the insufficiency determination in approximately two‐thirds of all safety concerns where ARIA was determined to be insufficient 37 . The Sentinel Innovation Center is working to improve the system's capabilities in this area through developing approaches to incorporate semi‐structured and unstructured EHR data into the SCDM 38 .…”

“…Inability to identify the health outcome of interest was one of the reasons for the insufficiency determination in approximately two-thirds of all safety concerns where ARIA was determined to be insufficient. 37 The Sentinel Innovation Center is working to improve the system's capabilities in this area through developing approaches to incorporate semi-structured and unstructured EHR data into the SCDM. 38 Initial steps in this field include creation of computable phenotypes by applying machine learning and natural language processing tools to laboratory data and clinical text, and validation of these algorithms using medical charts.…”

Quantifying Fit-for-Purpose in Real World Data: Data Grading and FitQ Scores

“…Similarly, the European Medicine Agency used a set of binary or three-point scales to grade multiple data sources but did not otherwise give comparison metrics. 6 The second gap is the need to extend fitness-for-purpose assessments to the new paradigm of modern multi-site collaboratives, such as the Food and Drug Administration's (FDA's) Sentinel Initiative 13 or the Observational Health Data Science Initiative (OHDSI) 14 . Current data grading frameworks primarily operate in the setting of bespoke RWE analyses.…”