Multimodal model with text and drug embeddings for adverse drug reaction classification

Sakhovskiy, Andrey; Tutubalina, Elena

doi:10.1016/j.jbi.2022.104182

Cited by 13 publications

(6 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multimodal language models jointly trained on English text and biological sequence data have already been used to identify protein-protein interactions [73], classify adverse reactions to drugs [74], and caption molecules [75]. The multimodal scientific language model Galactica was also trained on protein sequences [76].…”

Section: Discussionmentioning

confidence: 99%

OpenFold: Retraining AlphaFold2 yields new insights into its learning mechanisms and capacity for generalization

Ahdritz

Bouatta

Kadyan

et al. 2022

Preprint

127

109

View full text Add to dashboard Cite

AlphaFold2 revolutionized structural biology with the ability to predict protein structures with exceptionally high accuracy. Its implementation, however, lacks the code and data required to train new models. These are necessary to (i) tackle new tasks, like protein-ligand complex structure prediction, (ii) investigate the process by which the model learns, which remains poorly understood, and (iii) assess the model's generalization capacity to unseen regions of fold space. Here we report OpenFold, a fast, memory-efficient, and trainable implementation of AlphaFold2, and OpenProteinSet, the largest public database of protein multiple sequence alignments. We use OpenProteinSet to train OpenFold from scratch, fully matching the accuracy of AlphaFold2. Having established parity, we assess OpenFold's capacity to generalize across fold space by retraining it using carefully designed datasets. We find that OpenFold is remarkably robust at generalizing despite extreme reductions in training set size and diversity, including near-complete elisions of classes of secondary structure elements. By analyzing intermediate structures produced by OpenFold during training, we also gain surprising insights into the manner in which the model learns to fold proteins, discovering that spatial dimensions are learned sequentially. Taken together, our studies demonstrate the power and utility of OpenFold, which we believe will prove to be a crucial new resource for the protein modeling community.

show abstract

Section: Discussionmentioning

confidence: 99%

OpenFold: Retraining AlphaFold2 yields new insights into its learning mechanisms and capacity for generalization

Ahdritz

Bouatta

Kadyan

et al. 2022

Preprint

127

109

View full text Add to dashboard Cite

show abstract

“…• Sakhovskiy et al 42 proposes a multi-modal model with state-of-the-art BERT-based models for language understanding and molecular property prediction.…”

Section: Baseline Methodsmentioning

confidence: 99%

“…Lee et al 26 and Chowdhury et al 40 only provide the results on PSB2016. Meanwhile, Chen et al 41 and Sakhovskiy et al 42 only verify their model on SMM4H2018. However, our model and Li et al 11 perform experiments on both PSB2016 and SMM4H2018.…”

Section: Performance Comparison Of Our Methods and Other Existing Met...mentioning

confidence: 99%

Exploiting Question-Answer framework with multi-GRU to detect Adverse Drug Reaction on social media

Luo,

Yang

2024

Preprint

View full text Add to dashboard Cite

Adverse Drug Reactions (ADRs) stand out as a pressing challenge in public health and a critical aspect of drug discovery. The dilemma arises from the inherent impossibility of conducting a comprehensive evaluation of a drug before its market release, constrained by the limitations in scale and duration of clinical trials. Therefore, the post-marketing detection of ADRs in a timely and accurate manner becomes imperative. Adding to the complexity, a multitude of tweets harbor concealed information about adverse drug reactions, creating difficulties due to their concise, sporadic, and noisy content. To solve the problem, we regard ADR detection as a question-answer problem and introduces an innovative neural network framework with multiple GRU layers designed for extracting ADR-related information from tweets. The Von Mises–Fisher distribution (vMF distribution) is applied to derive keyword vectors through tweet sampling. An attention mecha-nism is employed to enhance the interaction between these keyword vectors and the word sequences within tweets. The credibility of word sequences is systematically evaluated based on the reliability of answer factors. To address concerns related to background information and training speed, we propose a quality assurance mechanism utilizing a GRU network due to its straightforward structure and efficient training capabilities. As a result of the training process, word sequences are mapped to a low-latitude vector space, generating corresponding answers. Experimental results obtained from two Twitter ADR datasets affirm that our Question-Answer Mechanism, leveraging multi-GRU architecture, significantly improves the accuracy of ADR detection in tweets. Our method achieved F1-scores of 81.3% and 73.3% on the two datasets, respectively, while consistently maintaining a higher recall.

show abstract

“…We use the annotated data of the ADE Extraction Task of the SMM4H 2020 shared task, which contains 1862 tweets, 1080 of which are positive for the presence of ADEs while the remaining 782 are negative. Similarly to previous works [56,57], we only use the annotated samples provided by the shared task (training and validation set) and not the blind test set for our analyses. The evaluation on the blind test set is available through the CodaLab platform 3 , however CodaLab allows for a limited number of test runs.…”

Section: Datasetsmentioning

confidence: 99%

Extensive evaluation of transformer-based architectures for adverse drug events extraction

Scaboro

Portelli

Chersoni

et al. 2023

Knowledge-Based Systems

View full text Add to dashboard Cite

Multimodal model with text and drug embeddings for adverse drug reaction classification

Cited by 13 publications

References 29 publications

OpenFold: Retraining AlphaFold2 yields new insights into its learning mechanisms and capacity for generalization

OpenFold: Retraining AlphaFold2 yields new insights into its learning mechanisms and capacity for generalization

Exploiting Question-Answer framework with multi-GRU to detect Adverse Drug Reaction on social media

Extensive evaluation of transformer-based architectures for adverse drug events extraction

Contact Info

Product

Resources

About