Learning from Context or Names? An Empirical Study on Neural Relation Extraction

Peng, Hao; Gao, Tianyu; Han, Xu; Lin, Yankai; Li, Peng; Liu, Zhiyuan; Sun, Maosong; Zhou, Jie

doi:10.18653/v1/2020.emnlp-main.298

Cited by 117 publications

(92 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some focus on obtaining better span representations, including entity mentions, via span-based pre-training Joshi et al, 2020;Kong et al, 2020;Ye et al, 2020). Others learn to extract relation-aware semantics from text by comparing the sentences that share the same entity pair or distantly supervised relation in KGs (Soares et al, 2019;Peng et al, 2020). However, these methods only consider either individual entities or within-sentence relations, which limits the performance in dealing with multiple entities and relations at document level.…”

Section: Related Workmentioning

confidence: 99%

ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning

Qin¹,

Lin²,

Takanobu³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

Self Cite

View full text Add to dashboard Cite

Pre-trained Language Models (PLMs) have shown superior performance on various downstream Natural Language Processing (NLP) tasks. However, conventional pre-training objectives do not explicitly model relational facts in text, which are crucial for textual understanding. To address this issue, we propose a novel contrastive learning framework ERICA to obtain a deep understanding of the entities and their relations in text. Specifically, we define two novel pre-training tasks to better understand entities and relations: (1) the entity discrimination task to distinguish which tail entity can be inferred by the given head entity and relation; (2) the relation discrimination task to distinguish whether two relations are close or not semantically, which involves complex relational reasoning. Experimental results demonstrate that ERICA can improve typical PLMs (BERT and RoBERTa) on several language understanding tasks, including relation extraction, entity typing and question answering, especially under low-resource settings. 1 * Corresponding author. 1 Our code and data are publicly available at https:// github.com/thunlp/ERICA. [1] Culiacán is a city in northwestern Mexico. [2] Culiacán is the capital of the state of Sinaloa. [3] Culiacán is also the seat of Culiacán Municipality. [4] It had an urban population of 785,800 in 2015 while 905,660 lived in the entire municipality. [5] While Culiacán Municipality has a total area of 4,758 k! ! , Culiacán itself is considerably smaller, measuring only. [6]Culiacán is a rail junction and is located on the Panamerican Highway that runs south to Guadalajara and Mexico City. [7] Culiacán is connected to the north with Los Mochis, and to the south with Mazatlán, Tepic.

show abstract

Section: Related Workmentioning

confidence: 99%

ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning

Qin¹,

Lin²,

Takanobu³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

Self Cite

View full text Add to dashboard Cite

show abstract

“…The success of supervised relation extraction methods (Bunescu and Mooney, 2005;Qian et al, 2008;Zeng et al, 2014;Zhou et al, 2016;Velikovi et al, 2018) depend heavily on large amount of annotated data. Due to the bottleneck of annotation cost, some weakly-supervied methods are proposed to learn the extraction model using distantly labeled datasets (Mintz et al, 2009;Hoffmann et al, 2011;Lin et al, 2016) or few-shot datasets (Han et al, 2018;Baldini Soares et al, 2019;Peng et al, 2020). However, these paradigms are still limited to predefined relation types from human definition or knowledge bases, which are usually unavailable in the open-domain scenario.…”

Section: Related Workmentioning

confidence: 99%

Element Intervention for Open Relation Extraction

Liu¹,

Yan²,

Lin³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

Open relation extraction aims to cluster relation instances referring to the same underlying relation, which is a critical step for general relation extraction. Current OpenRE models are commonly trained on the datasets generated from distant supervision, which often results in instability and makes the model easily collapsed. In this paper, we revisit the procedure of OpenRE from a causal view. By formulating OpenRE using a structural causal model, we identify that the above-mentioned problems stem from the spurious correlations from entities and context to the relation type.To address this issue, we conduct Element Intervention, which intervene on the context and entities respectively to obtain the underlying causal effects of them. We also provide two specific implementations of the interventions based on entity ranking and context contrasting. Experimental results on unsupervised relation extraction datasets show our method to outperform previous state-of-the-art methods and is robust across different datasets.

show abstract

“…In the NLP field, Dai and Lin (2017) proposed to use contrastive learning for image caption, and Clark et al (2020) trained a discriminative model for language representation learning. Recent literature (Peng et al, 2020) has also attempted to relate the contrastive pre-training scheme to classical supervised RE task. Different from our work, Peng et al (2020) aims to utilize abundant DS data and help classical supervised RE models learn a better relation representation, while our CIL focuses on learning an effective and efficient DSRE model under DS data noise.…”

Section: Sentencementioning

confidence: 99%

CIL: Contrastive Instance Learning Framework for Distantly Supervised Relation Extraction

Chen¹,

Shi²,

Tang³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

The journey of reducing noise from distant supervision (DS) generated training data has been started since the DS was first introduced into the relation extraction (RE) task. For the past decade, researchers apply the multiinstance learning (MIL) framework to find the most reliable feature from a bag of sentences. Although the pattern of MIL bags can greatly reduce DS noise, it fails to represent many other useful sentence features in the datasets. In many cases, these sentence features can only be acquired by extra sentence-level human annotation with heavy costs. Therefore, the performance of distantly supervised RE models is bounded. In this paper, we go beyond typical MIL framework and propose a novel Contrastive Instance Learning (CIL) framework. Specifically, we regard the initial MIL as the relational triple encoder and constraint positive pairs against negative pairs for each instance. Experiments demonstrate the effectiveness of our proposed framework, with significant improvements over the previous methods on NYT10, GDS and KBP.

show abstract

Learning from Context or Names? An Empirical Study on Neural Relation Extraction

Cited by 117 publications

References 31 publications

ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning

ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning

Element Intervention for Open Relation Extraction

CIL: Contrastive Instance Learning Framework for Distantly Supervised Relation Extraction

Contact Info

Product

Resources

About