Simultaneously Self-Attending to All Mentions for Full-Abstract
            Biological Relation Extraction

Verga, Patrick; Strubell, Emma; McCallum, Andrew

doi:10.18653/v1/n18-1080

Cited by 265 publications

(268 citation statements)

References 41 publications

Supporting

Mentioning

232

Contrasting

Unclassified

Order By: Relevance

“…GRU+Attn ) stacks a self-attention layer on top of GRU (Cho et al, 2014) and embedding layers; Bran (Verga et al, 2018) adopts a biaffine self-attention model to simultaneously extract the relations of all mention pairs. Both methods use only textual knowledge.…”

Section: Main Results On Biocreative VI Cprmentioning

confidence: 99%

Leveraging Dependency Forest for Neural Medical Relation Extraction

Song¹,

Zhang²,

Gildea³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Medical relation extraction discovers relations between entity mentions in text, such as research articles. For this task, dependency syntax has been recognized as a crucial source of features. Yet in the medical domain, 1best parse trees suffer from relatively low accuracies, diminishing their usefulness. We investigate a method to alleviate this problem by utilizing dependency forests. Forests contain many possible decisions and therefore have higher recall but more noise compared with 1-best outputs. A graph neural network is used to represent the forests, automatically distinguishing the useful syntactic information from parsing noise. Results on two biomedical benchmarks show that our method outperforms the standard tree-based methods, giving the state-of-the-art results in the literature.

show abstract

Section: Main Results On Biocreative VI Cprmentioning

confidence: 99%

Leveraging Dependency Forest for Neural Medical Relation Extraction

Song¹,

Zhang²,

Gildea³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

show abstract

“…Other more recent approaches include two-level reinforcement learning models (Takanobu et al, 2019), two layers of attention-based capsule network models , and self-attention with transformers (Verga et al, 2018). In particular, (Miwa and Sasaki, 2014;Katiyar and Cardie, 2016;Zhang et al, 2017;Zheng et al, 2017;Verga et al, 2018;Takanobu et al, 2019) also seek to jointly learn the entities and relations among them together. A large fraction of the past work focused on relations within a single sentences.…”

Section: Previous Workmentioning

confidence: 99%

Learning to Infer Entities, Properties and their Relations from Clinical Conversations

Wang

Tran³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Recently we proposed the Span Attribute Tagging (SAT) Model (Du et al., 2019) to infer clinical entities (e.g., symptoms) and their properties (e.g., duration). It tackles the challenge of large label space and limited training data using a hierarchical two-stage approach that identifies the span of interest in a tagging step and assigns labels to the span in a classification step.We extend the SAT model to jointly infer not only entities and their properties but also relations between them. Most relation extraction models restrict inferring relations between tokens within a few neighboring sentences, mainly to avoid high computational complexity. In contrast, our proposed Relation-SAT (R-SAT) model is computationally efficient and can infer relations over the entire conversation, spanning an average duration of 10 minutes. We evaluate our model on a corpus of clinical conversations. When the entities are given, the R-SAT outperforms baselines in identifying relations between symptoms and their properties by about 32% (0.82 vs 0.62 Fscore) and by about 50% (0.60 vs 0.41 Fscore) on medications and their properties. On the more difficult task of jointly inferring entities and relations, the R-SAT model achieves a performance of 0.34 and 0.45 for symptoms and medications respectively, which is significantly better than 0.18 and 0.35 for the baseline model. The contributions of different components of the model are quantified using ablation analysis.

show abstract

“…For the GAD and EU-ADR data sets, we use the train and test splits provided by Lee et al (2019). Also, for CDR, since the state-of-the-art results (Verga et al, 2018) are given at the abstract level, we re-run their proposed algorithm on our transformed sentence-level CDR data set, reporting results for a single model, without additional data (Verga et al (2018) reports also results when adding weakly labeled data). Let us first focus on the two unsupervised baselines.…”

Section: Data Sets and Setupmentioning

confidence: 99%

Deep Bidirectional Transformers for Relation Extraction without Supervision

Papanikolaou¹,

Roberts²,

Pierleoni³

2019

Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019)

View full text Add to dashboard Cite

We present a novel framework to deal with relation extraction tasks in cases where there is complete lack of supervision, either in the form of gold annotations, or relations from a knowledge base. Our approach leverages syntactic parsing and pre-trained word embeddings to extract few but precise relations, which are then used to annotate a larger corpus, in a manner identical to distant supervision. The resulting data set is employed to fine tune a pre-trained BERT model in order to perform relation extraction. Empirical evaluation on four data sets from the biomedical domain shows that our method significantly outperforms two simple baselines for unsupervised relation extraction and, even if not using any supervision at all, achieves slightly worse results than the state-of-the-art in three out of four data sets. Importantly, we show that it is possible to successfully fine tune a large pretrained language model with noisy data, as opposed to previous works that rely on gold data for fine tuning.

show abstract

Simultaneously Self-Attending to All Mentions for Full-Abstract Biological Relation Extraction

Cited by 265 publications

References 41 publications

Leveraging Dependency Forest for Neural Medical Relation Extraction

Leveraging Dependency Forest for Neural Medical Relation Extraction

Learning to Infer Entities, Properties and their Relations from Clinical Conversations

Deep Bidirectional Transformers for Relation Extraction without Supervision

Contact Info

Product

Resources

About