Machine Learning for Detection of Safety Signals From Spontaneous Reporting System Data: Example of Nivolumab and Docetaxel

Abstract

Introduction: Various methods have been implemented to detect adverse drug reaction (ADR) signals. However, the applicability of machine learning methods has not yet been fully evaluated.Objective: To evaluate the feasibility of machine learning algorithms in detecting ADR signals of nivolumab and docetaxel, new and old anticancer agents.Methods: We conducted a safety surveillance study of nivolumab and docetaxel using the Korea national spontaneous reporting database from 2009 to 2018. We constructed a novel input dataset for each study drug comprised of known ADRs that were listed in the drug labels and unknown ADRs. Given the known ADRs, we trained machine learning algorithms and evaluated predictive performance in generating safety signals of machine learning algorithms (gradient boosting machine [GBM] and random forest [RF]) compared with traditional disproportionality analysis methods (reporting odds ratio [ROR] and information component [IC]) by using the area under the curve (AUC). Each method then was implemented to detect new safety signals from the unknown ADR datasets.Results: Of all methods implemented, GBM achieved the best average predictive performance (AUC: 0.97 and 0.93 for nivolumab and docetaxel). The AUC achieved by each method was 0.95 and 0.92 (RF), 0.55 and 0.51 (ROR), and 0.49 and 0.48 (IC) for respective drug. GBM detected additional 24 and nine signals for nivolumab and 82 and 76 for docetaxel compared to ROR and IC, respectively, from the unknown ADR datasets.Conclusion: Machine learning algorithm based on GBM performed better and detected more new ADR signals than traditional disproportionality analysis methods.

Real-world data-based adverse drug reactions detection from the Korea Adverse Event Reporting System databases with electronic health records-based detection algorithm

“…Several studies to detect and predict ADRs risk using the KIDS-KD database have been actively conducted to date. [22][23][24][25] Soukavong et al 22 used a data mining approach to calculate the distribution of disproportionality between unknown drugs and ADRs. Yi et al 23 detected ADR signals in the cardiovascular system after using dipeptidyl peptidase-4 (DPP4) inhibitors.…”

“…Yi et al 23 detected ADR signals in the cardiovascular system after using dipeptidyl peptidase-4 (DPP4) inhibitors. Bae et al 24 applied machine-learning algorithms in consideration of known ADR to detect signals and compared the prediction performance using the existing disproportionality analysis (DPA) method. Various methods have been developed to detect ADR signals using the KIDS-KD, but the possibility of applying the machine-learning method has not yet been fully evaluated.…”

Aspects of Utilization and Limitations of Artificial Intelligence in Drug Safety

“…• Botsis et al [25] and Ball and Botsis [26] used network analysis approach to help in detecting signal from the safety surveillance data of FDA's Vaccine AERS • Harpaz et al [27] used clustering approach and discovered that a large proportion (41%) of clusters having associations (e.g. : chlorpromazine -hepatotoxicity, bosentan -hepatic steatosis, and methotrexate -pancytopenia) that are currently unrecognized but all of which are supported by older case reports • Ji-Hwan et al [28] compared ML algorithms with traditional DPA methods using dataset of known and unknown ADRs of Nivolumab and Docetaxel taken from Korea national spontaneous reporting database and found out that ML algorithms outperformed traditional DPA methods in detecting new ADR signals.…”

Identification of Biomarkers in Gynecologic Cancers: A Machine Learning Approach for Metabolomics

“…Boosting is an ensemble method that uses decision trees such as bagging and RF algorithms but has a different tting process. Based on previous model tting results, performance is continuously improved in the next model tting (Bae et al, 2020). Among diverse boosting algorithms, AdaBoost (for Adaptive Boosting) was used in this study.…”