Open Relation Extraction: Relational Knowledge Transfer from Supervised Data to Unsupervised Data

Wu, Ruidong; Yao, Yuan; Han, Xu; Xie, Rong; Liu, Zhiyuan; Lin, Feng; Lin, Leyu; Sun, Maosong

doi:10.18653/v1/d19-1021

Cited by 43 publications

(37 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There have been many efforts for open relation learning, including pattern extraction (Banko et al, 2007;Fader et al, 2011;Mausam et al, 2012;Del Corro and Gemulla, 2013;Angeli et al, 2015;Petroni et al, 2015;Stanovsky and Dagan, 2016;Mausam, 2016;Cui et al, 2018), relation discovery (Yao et al, 2011;Marcheggiani and Titov, 2016), relation clustering (Shinyama and Sekine, 2006;Elsahar et al, 2017;Wu et al, 2019), and data collection (Riloff et al, 1999;Etzioni et al, 2005;Pantel and Pennacchiotti, 2006;Rozenfeld and Feldman, 2008;Nakashole et al, 2011;Zhu et al, 2009;Gao et al, 2020). However, for continual relation learning, there are still only some preliminary explorations for it.…”

Section: Related Workmentioning

confidence: 99%

Continual Relation Learning via Episodic Memory Activation and Reconsolidation

Han¹,

Yi²,

Gao³

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

Self Cite

View full text Add to dashboard Cite

Continual relation learning aims to continually train a model on new data to learn incessantly emerging novel relations while avoiding catastrophically forgetting old relations. Some pioneering work has proved that storing a handful of historical relation examples in episodic memory and replaying them in subsequent training is an effective solution for such a challenging problem. However, these memorybased methods usually suffer from overfitting the few memorized examples of old relations, which may gradually cause inevitable confusion among existing relations. Inspired by the mechanism in human long-term memory formation, we introduce episodic memory activation and reconsolidation (EMAR) to continual relation learning. Every time neural models are activated to learn both new and memorized data, EMAR utilizes relation prototypes for memory reconsolidation exercise to keep a stable understanding of old relations. The experimental results show that EMAR could get rid of catastrophically forgetting old relations and outperform the state-of-the-art continual learning models. The code and datasets are released on https://github.com/thunlp/ ContinualRE.

show abstract

Section: Related Workmentioning

confidence: 99%

Continual Relation Learning via Episodic Memory Activation and Reconsolidation

Han¹,

Yi²,

Gao³

et al. 2020

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

Self Cite

View full text Add to dashboard Cite

show abstract

“…CNN Following [10], we take the CNN encoder as our first choice, which includes an embedding layer followed by a one-dimensional convolutional layer and a max-pooling layer. The model setting we used is the same as [10].…”

Section: Neural Encodersmentioning

confidence: 99%

“…Different from most other datasets, the entity pair of each instance in FewRel is unique, which makes the model unable to obtain shortcuts by memorizing the entities. Following [10], we choose 64 relations as the train set and randomly select 16 relations with 1600 instances as the test set; the remaining sentences are validation set. The other one is NYT+FB [19][20], which is built by distant supervision.…”

Section: Experiments Settingmentioning

confidence: 99%

“…Gao et al [9] propose a siamese network, which accumulates novel types with its few-shot instances. Besides these bootstrapping methods, another supervised scheme learns the similarity metrics from labeled instances and further transfer the relational knowledge to open-domain corpora, namely Relational Siamese Networks (RSNs) [10]. However, RSNs target at learning a similarity classifier rather than building relational representations directly.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

More: A Metric Learning Based Framework for Open-Domain Relation Extraction

Wang

Lou

Zhang

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Open relation extraction (OpenRE) is the task of extracting relation schemes from open-domain corpora. Most existing OpenRE methods either do not fully benefit from high-quality labeled corpora or can not learn semantic representation directly, affecting downstream clustering efficiency. To address these problems, in this work, we propose a novel learning framework named MORE (Metric learning-based Open Relation Extraction). The framework utilizes deep metric learning to obtain rich supervision signals from labeled data and drive the neural model to learn semantic relational representation directly. Experiments result in two real-world datasets show that our method outperforms other state-of-theart baselines. Our source code is available on Github 1 .

show abstract

“…This method naturally makes good use of the context, dynamically produces word vector representations, and trains MLM such as denoising targets on large-scale corpora. The resulting representation is very helpful for downstream tasks and solves the polysemy problem [10] [11].…”

Section: Introductionmentioning

confidence: 99%

BERT-Based Chinese Relation Extraction for Public Security

Hou

Yao

et al. 2020

IEEE Access

View full text Add to dashboard Cite

The past few years have witnessed some public safety incidents occurring around the world. With the advent of the big data era, effectively extracting public security information from the internet has become of great significance. Up to hundreds of TBs of data are injected into the network every second, and thus it is impossible to process them manually. Natural Language Processing (NLP) is dedicated to the development of an intelligent system for effective text information mining. By analysing the text and quickly extracting the relationships between the relevant entities, NLP can establish the knowledge graph (KG) of public security, which lays the foundation for safety case analysis, information monitoring, and activity tracking and locating. One of the current pre-training relation extraction models is the Word2Vec model. The Word2vec model is single mapped, and it produces a static, single representation of the words in sentences. Then, the BERT model considers contextual information and provides more dynamic, richer vector representations of generated words. Therefore, in this paper, we propose a Bidirectional Encoder Representation from Transformers (BERT) based on the Chinese relation extraction algorithm for public security, which can effectively mine security information. The BERT model is obtained by training the Masked Language Model and predicting the next sentence task, which is based on the Transformer Encoder and the main model structure is the stacked Transformers. Extensive simulations are conducted to evaluate our proposed algorithm in comparison to some state-of-the-art schemes.

show abstract

Open Relation Extraction: Relational Knowledge Transfer from Supervised Data to Unsupervised Data

Cited by 43 publications

References 16 publications

Continual Relation Learning via Episodic Memory Activation and Reconsolidation

Continual Relation Learning via Episodic Memory Activation and Reconsolidation

More: A Metric Learning Based Framework for Open-Domain Relation Extraction

BERT-Based Chinese Relation Extraction for Public Security

Contact Info

Product

Resources

About