SENT: Sentence-level Distant Relation Extraction via Negative Training

Abstract

Distant supervision for relation extraction provides uniform bag labels for each sentence inside the bag, while accurate sentence labels are important for downstream applications that need the exact relation type. Directly using bag labels for sentence-level training will introduce much noise, thus severely degrading performance. In this work, we propose the use of negative training (NT), in which a model is trained using complementary labels regarding that "the instance does not belong to these complementary labels". Since the probability of selecting a true label as a complementary label is low, NT provides less noisy information. Furthermore, the model trained with NT is able to separate the noisy data from the training data. Based on NT, we propose a sentence-level framework, SENT, for distant relation extraction. SENT not only filters the noisy data to construct a cleaner dataset, but also performs a relabeling process to transform the noisy data into useful training data, thus further benefiting the model's performance. Experimental results show the significant improvement of the proposed method over previous methods on sentence-level evaluation and de-noise effect.

Abstains from Prediction: Towards Robust Relation Extraction in Real World

“…Supervised/Distantly supervised relation extraction is oriented at predefined relational types. Researchers have explored different network architectures (Zhang et al, 2018), training strategies (Ma et al, 2021) and external information (Zhang et al, 2019) (Fader et al, 2011), clustering (Zhao et al, 2021) are used to deal with relations without pre-specified schemas. Different from them, we consider a more general scenario, in which known and unknown relations are mixed in the input.…”

“…Different optimization objectives such as large margin loss (Lin and Xu, 2019), gaussian mixture loss (Yan et al, 2020) are adopted to learn more discriminative representations to facilitate anomaly detection. Recently, Zhang et al (2021) propose to learn the adaptive decision boundary (ADB) that serves as the basis for judging outliers.…”

“…The existing works mainly focus on how to alleviate the noise generated in the automatic annotation. Common approaches include selecting informative instances (Lin et al, 2016), incorporating extra information (Zhang et al, 2019), and designing sophisticated training (Ma et al, 2021).…”

Diversifying Neural Dialogue Generation via Negative Distillation

“…of models to meet the requirement of diverse responses instead of MLE, such as MMI (Li et al, 2016), AdaLabel (Wang et al, 2021), and IAT (Zhou et al, 2021). Besides, several studies (Kulikov et al, 2019;Holtzman et al, 2020) proposed more advanced decoding strategies to alleviate the problem of generic responses.…”

“…However, inspired by negative training (Kim et al, 2019;Ma et al, 2021), we argue that it is also necessary to tell the dialogue model what not to say. To alleviate the problem of generic responses, He and Glass (2020) negatively updates the parameters when identifying the high-frequency responses.…”