Reliably Filter Drug-Induced Liver Injury Literature With Natural Language Processing and Conformal Prediction

Zhan, Xianghao; Wang, Fanjin; Gevaert, Olivier

doi:10.1109/jbhi.2022.3193365

Cited by 10 publications

(10 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Critical Assessment of Massive Data Analysis (CAMDA) 2022 in collaboration with the Intelligent Systems for Molecular Biology (ISMB) hosted the Literature AI for Drug Induced Liver Injury (DILI) challenge [1]. A curated dataset, consisting of 277,016 DILI annotated papers, was downloaded from the CAMDA website.…”

mentioning

confidence: 99%

Automatic identification of drug-induced liver injury literature using natural language processing and machine learning methods

Tannenbaum

Deasy

2022

Preprint

View full text Add to dashboard Cite

Drug-induced liver injury (DILI) is an adverse hepatic drug reaction that can potentially lead to life-threatening liver failure. Previously published work in the scientific literature on DILI has provided valuable insights for the understanding of hepatotoxicity as well as drug development. However, the manual search of scientific literature in PubMed is laborious. Natural language processing (NLP) techniques have been developed to decipher and understand the meaning of human language by extracting useful information from unstructured text data. In particular, NLP along with artificial intelligence (AI) / machine learning (ML) techniques may allow automatic processing of the DILI literature, but useful methods are yet to be demonstrated. To address this challenge, we have developed an integrated NLP/ML classification model to identify DILI-related literature using only paper titles and abstracts. We used 14,203 publications provided by the Critical Assessment of Massive Data Analysis (CAMDA) challenge, employing word vectorization techniques in NLP coupled with machine learning methods. Classification modeling was performed using 2/3 of the data for training and the remainder for testing in internal validation. The best performance was achieved using a linear support vector machine (SVM) model that combined vectors derived from term frequency-inverse document frequency (TF-IDF) and Word2Vec, achieving an accuracy of 95.0% and an F1-score of 95.0%. The final SVM model built using all 14,203 publications was tested on independent datasets, resulting in accuracies of 92.5%, 96.3%, and 98.3%, and F1-scores of 93.5%, 86.1%, and 75.6% for three test sets (T1-T3). The SVM model was tested on four external validation sets (V1-V4), resulting in accuracies of 92.0%, 96.2%, 98.3%, and 93.1%, and F1-scores of 92.4%, 82.9%, 75.0%, and 93.3%.

show abstract

mentioning

confidence: 99%

Automatic identification of drug-induced liver injury literature using natural language processing and machine learning methods

Tannenbaum

Deasy

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…242 The model using TF-IDF and logistic regression achieved an accuracy of 0.957 with an ensemble model fine-tuned to lower false-negative cases, achieving an accuracy of 0.954 and an F1 score of 0.955. 242 The ensemble model identified important words in positive/negative predictions, providing researchers with a rapid filter for DILI-related literature. Overall, these research studies aim to assist regulatory activities in evaluating drug safety and identifying potential DILI risks, making it indirectly related to active drug toxicity prediction efforts.…”

Section: Normalizing Flow and Diffusionmentioning

confidence: 99%

“…The study demonstrated that the model could capture the semantic meanings of complex text in drug labeling and was portable across agencies, indicating the potential for using AI technologies to modernize and advance regulatory science. Another similar study used BERT-powered NLP to automatically filter out DILI literature from around 28,000 papers provided by the CAMDA challenge . The model using TF-IDF and logistic regression achieved an accuracy of 0.957 with an ensemble model fine-tuned to lower false-negative cases, achieving an accuracy of 0.954 and an F1 score of 0.955 .…”

Section: Deep Learning In Predictive Drug Toxicity Studiesmentioning

confidence: 99%

“…Another similar study used BERT-powered NLP to automatically filter out DILI literature from around 28,000 papers provided by the CAMDA challenge . The model using TF-IDF and logistic regression achieved an accuracy of 0.957 with an ensemble model fine-tuned to lower false-negative cases, achieving an accuracy of 0.954 and an F1 score of 0.955 . The ensemble model identified important words in positive/negative predictions, providing researchers with a rapid filter for DILI-related literature.…”

Section: Deep Learning In Predictive Drug Toxicity Studiesmentioning

confidence: 99%

See 1 more Smart Citation

A Review on the Recent Applications of Deep Learning in Predictive Drug Toxicological Studies

Sinha¹,

Ghosh²,

Sil

2023

Chem. Res. Toxicol.

View full text Add to dashboard Cite

Drug toxicity prediction is an important step in ensuring patient safety during drug design studies. While traditional preclinical studies have historically relied on animal models to evaluate toxicity, recent advances in deep-learning approaches have shown great promise in advancing drug safety science and reducing animal use in preclinical studies. However, deep-learning-based approaches also face challenges in handling large biological data sets, model interpretability, and regulatory acceptance. In this review, we provide an overview of recent developments in deep-learning-based approaches for predicting drug toxicity, highlighting their potential advantages over traditional methods and the need to address their limitations. Deep-learning models have demonstrated excellent performance in predicting toxicity outcomes from various data sources such as chemical structures, genomic data, and high-throughput screening assays. The potential of deep learning for automated feature engineering is also discussed. This review emphasizes the need to address ethical concerns related to the use of deep learning in drug toxicity studies, including the reduction of animal use and ensuring regulatory acceptance. Furthermore, emerging applications of deep learning in drug toxicity prediction, such as predicting drug–drug interactions and toxicity in rare subpopulations, are highlighted. The integration of deep-learning-based approaches with traditional methods is discussed as a way to develop more reliable and efficient predictive models for drug safety assessment, paving the way for safer and more effective drug discovery and development. Overall, this review highlights the critical role of deep learning in predictive toxicology and drug safety evaluation, emphasizing the need for continued research and development in this rapidly evolving field. By addressing the limitations of traditional methods, leveraging the potential of deep learning for automated feature engineering, and addressing ethical concerns, deep-learning-based approaches have the potential to revolutionize drug toxicity prediction and improve patient safety in drug discovery and development.

show abstract

“…In our current DILI study, binary classification has been considered. Logistic regression is exemplified for DILI classification [18,19]. Once again, if 𝑥 be the input features of dimension 𝑛, and 𝑦 be the binary class with class labels 0 and 1.…”

Section: Model Selection and Mathematical Analysismentioning

confidence: 99%

Machine Learning to Predict Drug-Induced Liver Injury and its Validation on Failed Drug Candidates in Development

Mostafa,

Howle,

Chen

2024

Preprint

View full text Add to dashboard Cite

Drug-induced liver injury (DILI) remains a significant challenge for the pharmaceutical industry and regulatory organizations. Despite a plethora of toxicological research aimed at estimating the risk of DILI, the efficacy of these techniques in predicting DILI in humans has remained limited. This has prompted the exploration of new approaches and procedures to improve the prediction accuracy of DILI risk for drug candidates in development. This study aimed to address this gap by leveraging a large human dataset to develop machine learning models for assessing DILI risk. The performance of the developed prediction models was extensively evaluated using a 10-fold cross-validation approach and two external test sets. Our study revealed that the Random Forest (RF) and MultiLayer Perceptron (MLP) models emerged as among the most effective in predicting DILI. RF outperformed other machine learning strategies, reaching an average prediction accuracy of 63.10% during the cross-validation, while the MLP achieved the highest Matthews Correlation Coefficient (MCC) of 0.245. These two models were further validated externally by a set of drug candidates that failed in clinical development due to DILI. Both models accurately predicted 90.9% of the toxic drug candidates in the external validation. Our study suggests that in silico machine learning approaches have the potential to significantly enhance the identification of DILI liabilities associated with drug candidates in development.

show abstract

Reliably Filter Drug-Induced Liver Injury Literature With Natural Language Processing and Conformal Prediction

Cited by 10 publications

References 34 publications

Automatic identification of drug-induced liver injury literature using natural language processing and machine learning methods

Automatic identification of drug-induced liver injury literature using natural language processing and machine learning methods

A Review on the Recent Applications of Deep Learning in Predictive Drug Toxicological Studies

Machine Learning to Predict Drug-Induced Liver Injury and its Validation on Failed Drug Candidates in Development

Contact Info

Product

Resources

About