Incorporating Template-based Contrastive Learning into cognitively inspired, low resource relation extraction

Abstract

Background From an unstructured text, Relation Extraction (RE) predicts semantic relationships between pairs of entities. The process of labeling tokens and phrases can be very expensive and require a great deal of time and effort. The Low-Resource Relation Extraction (LRE) problem comes into being, is challenging since there are only a limited number of annotated sentences available. Recent research has focused on minimizing the cross-entropy loss between pseudo labels and ground truth or on using external knowledge to make annotations for unlabeled data. Existing methods, however, fail to take into account the semantics of relation types and the information hidden within different relation groups. Method By drawing inspiration from the process of human interpretation of unstructured documents, we introduce Template-based Contrastive Learning (TempCL). Through the use of template, we limit the model's attention to the semantic information that is contained in a relation. Then we employ a constrastive learning strategy using both group-wise and instance-wise perspectives to leverage shared semantic information within the same relation type to achieve a more coherent semantic representation. Particularly, the proposed group-wise contrastive learning minimizes the discrepancy between the template and original sentences in the same label group and maximizes the difference between those from separate label groups under limited annotation settings. Results Our experiment results on two public datasets show that our model TempCL achieves state-of-the-art results for low resource relation extraction in comparison to baselines. The relative error reductions range from 0.68 to 1.32%. Conclusion Our model encourages the feature to be aligned with both the original and template sentences. Using two contrastive losses, we exploit shared semantic information underlying sentences (both original and template) that have the same relation type. We demonstrate that our method reduces the noise caused by tokens that are unrelated and constrains the model's attention to the tokens that are related.