Exploit a Multi-head Reference Graph for Semi-supervised Relation Extraction

Li, Wanli; Qian, Tieyun; Chen, Xu; Tang, Kejian; Zhan, Shaohui; Zhan, Tao

doi:10.1109/ijcnn52387.2021.9534434

Cited by 12 publications

(7 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For performance comparison, we select Self-Training [15], Mean-Teacher [25], DualRE [26], RE-Ensemble [26], MRefG [16], MetaSRE [7], GradLRE [8], SelfLRE [27] and UG-MCT [28] as our baselines. The upper bound for our models is the pre-trained model BERT [22] (denoted as BERT w gold labels).…”

Section: Baselinesmentioning

confidence: 99%

“…The Distant Supervision method [5,6] exploits external knowledge bases(KBs) to annotate unlabeled data and uses them as training data. The Self-Training method [7,8,15,16] assigns pseudo labels and uses them to incrementally improve the model's generalization. In spite of their good results, the existing models ignore the semantics of labels and the information shared between sentences.…”

Section: Low Resource Relation Extractoin Low Resource Relation Extra...mentioning

confidence: 99%

“…(4) RE-Ensemble [26] replaces the retrieval module in the proposed DualRE framework with the same prediction module. (5) MRefG [16] constructs reference graphs to make semantically or lexically connections between the unlabeled data and the labeled data. The reference includes entity reference, verb reference and semantic reference.…”

Section: Baselinesmentioning

confidence: 99%

See 2 more Smart Citations

Incorporating Template-based Contrastive Learning into cognitively inspired, low resource relation extraction

Zheng,

Luu

2024

Preprint

View full text Add to dashboard Cite

Background From an unstructured text, Relation Extraction (RE) predicts semantic relationships between pairs of entities. The process of labeling tokens and phrases can be very expensive and require a great deal of time and effort. The Low-Resource Relation Extraction (LRE) problem comes into being, is challenging since there are only a limited number of annotated sentences available. Recent research has focused on minimizing the cross-entropy loss between pseudo labels and ground truth or on using external knowledge to make annotations for unlabeled data. Existing methods, however, fail to take into account the semantics of relation types and the information hidden within different relation groups. Method By drawing inspiration from the process of human interpretation of unstructured documents, we introduce Template-based Contrastive Learning (TempCL). Through the use of template, we limit the model's attention to the semantic information that is contained in a relation. Then we employ a constrastive learning strategy using both group-wise and instance-wise perspectives to leverage shared semantic information within the same relation type to achieve a more coherent semantic representation. Particularly, the proposed group-wise contrastive learning minimizes the discrepancy between the template and original sentences in the same label group and maximizes the difference between those from separate label groups under limited annotation settings. Results Our experiment results on two public datasets show that our model TempCL achieves state-of-the-art results for low resource relation extraction in comparison to baselines. The relative error reductions range from 0.68 to 1.32%. Conclusion Our model encourages the feature to be aligned with both the original and template sentences. Using two contrastive losses, we exploit shared semantic information underlying sentences (both original and template) that have the same relation type. We demonstrate that our method reduces the noise caused by tokens that are unrelated and constrains the model's attention to the tokens that are related.

show abstract

Section: Baselinesmentioning

confidence: 99%

Section: Low Resource Relation Extractoin Low Resource Relation Extra...mentioning

confidence: 99%

See 1 more Smart Citation

Incorporating Template-based Contrastive Learning into cognitively inspired, low resource relation extraction

Zheng,

Luu

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…We compare REMix with three state-of-the-art models that are representative of the existing class of methods for SSRE: MRefG (Li et al, 2021), MetaSRE (Hu et al, 2021a), and GradLRE (Hu et al, 2021b). MRefG leverages the unlabelled data by semantically or lexically connecting them to labelled data by constructing reference graphs, such as entity reference or verb reference.…”

Section: Baselines and Implementation Detailsmentioning

confidence: 99%

Semi-supervised Relation Extraction via Data Augmentation and Consistency-training

Teru

2023

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics

View full text Add to dashboard Cite

Due to the semantic complexity of the Relation extraction (RE) task, obtaining high-quality human labelled data is an expensive and noisy process. To improve the sample efficiency of the models, semi-supervised learning (SSL) methods aim to leverage unlabelled data in addition to learning from limited labelled data points. Recently, strong data augmentation combined with consistency-based semi-supervised learning methods have advanced the state of the art in several SSL tasks. However, adapting these methods to the RE task has been challenging due to the difficulty of data augmentation for RE. In this work, we leverage the recent advances in controlled text generation to perform high quality data augmentation for the RE task. We further introduce small but significant changes to model architecture that allows for generation of more training data by interpolating different data points in their latent space. These data augmentations along with consistency training result in very competitive results for semi-supervised relation extraction on four benchmark datasets.

show abstract

“…Following (Rosenberg, Hebert, and Schneiderman 2005;Tarvainen and Valpola 2017;Lin et al 2019b;Li et al 2021;Hu et al 2021a,b), we adopt F1 score as the evaluation metric and precision and recall as auxiliary metrics. For data settings, we follow (Lin et al 2019b;Hu et al 2021a,b) to divide the training set into labeled and unlabeled sets.…”

Section: Experimental Settingsmentioning

confidence: 99%

Competition or Cooperation? Exploring Unlabeled Data via Challenging Minimax Game for Semi-supervised Relation Extraction

Feng

et al. 2023

AAAI

View full text Add to dashboard Cite

Semi-Supervised Relation Extraction aims at learning well-performed RE models with limited labeled and large-scale unlabeled data. Existing methods mainly suffer from semantic drift and insufficient supervision, which severely limit the performance. To address these problems, recent work tends to design dual modules to work cooperatively for mutual enhancement. However, the consensus of two modules greatly restricts the model from exploring diverse relation expressions in unlabeled set, which hinders the performance as well as model generalization. To tackle this problem, in this paper, we propose a novel competition-based method AdvSRE. We set up a challenging minimax game on unlabeled data between two modules, Generator and Discriminator, and assign them with conflicting objectives. During the competition game, one module may find any possible chance to beat the other, which develops two modules' abilities until relation expressions cannot be further explored. To exploit label information, Discriminator is further asked to predict specific relation for each sentence. Experiment results on two benchmarks show new state-of-the-art performance over baselines, demonstrating the effectiveness of proposed AdvSRE.

show abstract

Exploit a Multi-head Reference Graph for Semi-supervised Relation Extraction

Cited by 12 publications

References 26 publications

Incorporating Template-based Contrastive Learning into cognitively inspired, low resource relation extraction

Incorporating Template-based Contrastive Learning into cognitively inspired, low resource relation extraction

Semi-supervised Relation Extraction via Data Augmentation and Consistency-training

Competition or Cooperation? Exploring Unlabeled Data via Challenging Minimax Game for Semi-supervised Relation Extraction

Contact Info

Product

Resources

About