Are Noisy Sentences Useless for Distant Supervised Relation Extraction?

Shang, Yu-Ming; Huang, He; Mao, Xian-Ling; Sun, Xin; Wei, Wei

doi:10.1609/aaai.v34i05.6407

Cited by 33 publications

(22 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In MIL, the training and testing processes are performed at the bag level, where a bag contains noisy sentences mentioning the same entity pair but possibly not describing the same relation. Studies using MIL can be broadly classified into two categories: 1) the soft de-noise methods that leverage soft weights to differentiate the influence of each sentence (Lin et al, 2016;Han et al, 2018c;Li et al, 2020;Hu et al, 2019a;Ye and Ling, 2019;Yuan et al, 2019a,b); 2) the hard de-noise methods that remove noisy sentences from the bag (Zeng et al, 2015;Qin et al, 2018;Han et al, 2018a;Shang, 2019). However, these bag-level approaches fail to map each sentence inside bags with explicit sentence labels.…”

Section: Bag Labels (We Need) Sentence Labelsmentioning

confidence: 99%

SENT: Sentence-level Distant Relation Extraction via Negative Training

Ma¹,

Gui²,

Li³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

Distant supervision for relation extraction provides uniform bag labels for each sentence inside the bag, while accurate sentence labels are important for downstream applications that need the exact relation type. Directly using bag labels for sentence-level training will introduce much noise, thus severely degrading performance. In this work, we propose the use of negative training (NT), in which a model is trained using complementary labels regarding that "the instance does not belong to these complementary labels". Since the probability of selecting a true label as a complementary label is low, NT provides less noisy information. Furthermore, the model trained with NT is able to separate the noisy data from the training data. Based on NT, we propose a sentence-level framework, SENT, for distant relation extraction. SENT not only filters the noisy data to construct a cleaner dataset, but also performs a relabeling process to transform the noisy data into useful training data, thus further benefiting the model's performance. Experimental results show the significant improvement of the proposed method over previous methods on sentence-level evaluation and de-noise effect.

show abstract

Section: Bag Labels (We Need) Sentence Labelsmentioning

confidence: 99%

SENT: Sentence-level Distant Relation Extraction via Negative Training

Ma¹,

Gui²,

Li³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

show abstract

“…Apart from that, methods treat the identified potentially noisy samples differently. They are either kept for further training with reduced weights (Jat et al, 2018;He et al, 2020), corrected (Shang, 2019) or eliminated (Qin et al, 2018). Thus, denoising methods vary significantly depending on the data and task, what makes the creation of a platform for comparison crucial.…”

Section: Weak Supervisionmentioning

confidence: 99%

Knodle: Modular Weakly Supervised Learning with PyTorch

Sedova¹,

Stephan²,

Speranskaya³

et al. 2021

Proceedings of the 6th Workshop on Representation Learning for NLP (RepL4NLP-2021)

View full text Add to dashboard Cite

Strategies for improving the training and prediction quality of weakly supervised machine learning models vary in how much they are tailored to a specific task or integrated with a specific model architecture. In this work, we introduce Knodle, a software framework that treats weak data annotations, deep learning models, and methods for improving weakly supervised training as separate, modular components. This modularization gives the training process access to fine-grained information such as data set characteristics, matches of heuristic rules, or elements of the deep learning model ultimately used for prediction. Hence, our framework can encompass a wide range of training methods for improving weak supervision, ranging from methods that only look at correlations of rules and output classes (independently of the machine learning model trained with the resulting labels), to those that harness the interplay of neural networks and weakly labeled data. We illustrate the benchmarking potential of the framework with a performance comparison of several reference implementations on a selection of datasets that are already available in Knodle.

show abstract

“…In the field of distantly supervised relation extraction, the multi-instance learning framework is introduced to handle the label noise of DS. Recently, MIL has become a common paradigm for DSRE and many efforts have been made for further improvements (Lin et al, 2016;Qin et al, 2018;Ye and Ling, 2019;Huang and Du, 2019;Shang et al, 2020). In these MIL frameworks, sentences are first encoded by handcrafted features (Mintz et al, 2009;Hoffmann et al, 2011) or neural networks.…”

Section: Related Workmentioning

confidence: 99%

“…We evaluate our model on the widely used DSRE dataset -NYT (Mintz et al, 2009), which aligns Freebase (Bollacker et al, 2008) (Huang and Du, 2019;Shang et al, 2020) since then adopt the correct dataset as the benchmark. In order to ensure the fairness and scientificity of our experiments, we use the original dataset release and employ the popular RE toolkit OpenNRE (Han et al, 2019) in our study .…”

Section: Datasetmentioning

confidence: 99%

See 1 more Smart Citation

Empower Distantly Supervised Relation Extraction with Collaborative Adversarial Training

Chen¹,

Shi²,

Liu³

et al. 2021

Preprint

View full text Add to dashboard Cite

With recent advances in distantly supervised (DS) relation extraction (RE), considerable attention is attracted to leverage multi-instance learning (MIL) to distill high-quality supervision from the noisy DS. Here, we go beyond label noise and identify the key bottleneck of DS-MIL to be its low data utilization: as highquality supervision being refined by MIL, MIL abandons a large amount of training instances, which leads to a low data utilization and hinders model training from having abundant supervision. In this paper, we propose collaborative adversarial training to improve the data utilization, which coordinates virtual adversarial training (VAT) and adversarial training (AT) at different levels. Specifically, since VAT is label-free, we employ the instance-level VAT to recycle instances abandoned by MIL. Besides, we deploy AT at the bag-level to unleash the full potential of the high-quality supervision got by MIL. Our proposed method brings consistent improvements (∼ 5 absolute AUC score) to the previous state of the art, which verifies the importance of the data utilization issue and the effectiveness of our method.

show abstract

Are Noisy Sentences Useless for Distant Supervised Relation Extraction?

Cited by 33 publications

References 12 publications

SENT: Sentence-level Distant Relation Extraction via Negative Training

SENT: Sentence-level Distant Relation Extraction via Negative Training

Knodle: Modular Weakly Supervised Learning with PyTorch

Empower Distantly Supervised Relation Extraction with Collaborative Adversarial Training

Contact Info

Product

Resources

About