Extracting chemical–protein relations with ensembles of SVM and deep learning models

Peng, Yifan; Rios, Anthony; Kavuluru, Ramakanth; Lu, Zhiyong

doi:10.1093/database/bay073

Cited by 113 publications

(83 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…; BC5CDR-disease, BC5CDR-chem (Yoon et al, 2018); ShARe/CLEFE (Leaman et al, 2015); DDI (Zhang et al, 2018). Chem-Prot (Peng et al, 2018); i2b2 (Rink et al, 2011); HoC (Du et al, 2019); MedNLI (Romanov and Shivade, 2018). P: PubMed, P+M: PubMed + MIMIC-III For named entity recognition, we used a Bi-LSTM-CRF implementation as a sequence tagger Si et al, 2019;Lample et al, 2016).…”

Section: Fine-tuning With Elmomentioning

confidence: 99%

Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets

Peng

Yan

2019

Proceedings of the 18th BioNLP Workshop and Shared Task

Self Cite

587

462

View full text Add to dashboard Cite

Inspired by the success of the General Language Understanding Evaluation benchmark, we introduce the Biomedical Language Understanding Evaluation (BLUE) benchmark to facilitate research in the development of pre-training language representations in the biomedicine domain. The benchmark consists of five tasks with ten datasets that cover both biomedical and clinical texts with different dataset sizes and difficulties. We also evaluate several baselines based on BERT and ELMo and find that the BERT model pre-trained on PubMed abstracts and MIMIC-III clinical notes achieves the best results. We make the datasets, pre-trained models, and codes publicly available at https://github.com/ ncbi-nlp/BLUE_Benchmark.

show abstract

Section: Fine-tuning With Elmomentioning

confidence: 99%

Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets

Peng

Yan

2019

Proceedings of the 18th BioNLP Workshop and Shared Task

Self Cite

587

462

View full text Add to dashboard Cite

show abstract

“…Two of the top three teams (Peng et al (48) with an F1 score of 0.641 and Mehryary et al (49) with an F1 score of 0.609) used ensembles of support vector machine (SVM) and DNNs. Peng et al developed a rich feature SVM including words, POS tags, chunk types, contextual words of entities, distance, selected keywords and shortest dependency path features.…”

Section: Discussionmentioning

confidence: 99%

Extracting chemical–protein relations using attention-based neural networks

et al. 2018

View full text Add to dashboard Cite

Relation extraction is an important task in the field of natural language processing. In this paper, we describe our approach for the BioCreative VI Task 5: text mining chemical–protein interactions. We investigate multiple deep neural network (DNN) models, including convolutional neural networks, recurrent neural networks (RNNs) and attention-based (ATT-) RNNs (ATT-RNNs) to extract chemical–protein relations. Our experimental results indicate that ATT-RNN models outperform the same models without using attention and the ATT-gated recurrent unit (ATT-GRU) achieves the best performing micro average F1 score of 0.527 on the test set among the tested DNNs. In addition, the result of word-level attention weights also shows that attention mechanism is effective on selecting the most important trigger words when trained with semantic relation labels without the need of semantic parsing and feature engineering. The source code of this work is available at https://github.com/ohnlp/att-chemprot.

show abstract

“…Many biomedical relation extraction systems have often relied hand-crafted linguistic features (Gu et al, 2016;Peng et al, 2016) but recently also convolutional neural networks (Nguyen and Verspoor, 2018;Choi, 2018), LSTM (Li et al, 2017;Sahu and Anand, 2018) or a combination of machine learning models and neuralnetwork-based encoders (Zhang et al, 2018;Peng et al, 2018). A recent paper (Verga et al, 2018) achieves state-of-the-art results on biomedical relation classification for chemically-induced diseases (CDR (Li et al, 2016)) and ChemProt (CPR (Krallinger M., 2017)), by using a Transformer encoder (Vaswani et al, 2017) and end-to-end Named Entity Recognition and relation extraction, without, however, leveraging transformer-based language model pre-training.…”

Section: Related Workmentioning

confidence: 99%

Biomedical relation extraction with pre-trained language representations and minimal task-specific architecture

Thillaisundaram¹,

Togia²

2019

Proceedings of the 5th Workshop on BioNLP Open Shared Tasks

View full text Add to dashboard Cite

This paper presents our participation in the AGAC Track from the 2019 BioNLP Open Shared Tasks. We provide a solution for Task 3, which aims to extract "gene -function change -disease" triples, where "gene" and "disease" are mentions of particular genes and diseases respectively and "function change" is one of four pre-defined relationship types. Our system extends BERT (Devlin et al., 2018), a state-of-the-art language model, which learns contextual language representations from a large unlabelled corpus and whose parameters can be fine-tuned to solve specific tasks with minimal additional architecture. We encode the pair of mentions and their textual context as two consecutive sequences in BERT, separated by a special symbol. We then use a single linear layer to classify their relationship into five classes (four pre-defined, as well as 'no relation'). Despite considerable class imbalance, our system significantly outperforms a random baseline while relying on an extremely simple setup with no specially engineered features.

show abstract

Extracting chemical–protein relations with ensembles of SVM and deep learning models

Cited by 113 publications

References 16 publications

Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets

Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets

Extracting chemical–protein relations using attention-based neural networks

Biomedical relation extraction with pre-trained language representations and minimal task-specific architecture

Contact Info

Product

Resources

About