Learn from Relation Information: Towards Prototype Representation Rectification for Few-Shot Relation Extraction

Liu, Yang; Hu, Jinpeng; Xiang, Wei; Chang, Tsung-Hui

doi:10.18653/v1/2022.findings-naacl.139

Cited by 7 publications

(4 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[28] employs instruction finetuning to comprehend and process task-specific instructions, including both main and auxiliary tasks. The PRM approach relies on human-generated descriptions that necessitate expert input and effort, proving less adaptable to new classes [29]. [30] extracts entity type-related features based on mutual information criteria and generates a unique prompt for each unseen example by selecting relevant entity type-related features.…”

Section: Related Workmentioning

confidence: 99%

Fighting Against the Repetitive Training and Sample Dependency Problem in Few-Shot Named Entity Recognition

Tian,

Yin,

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Few-shot named entity recognition (NER) systems recognize entities using a few labeled training examples. The general pipeline consists of a span detector to identify entity spans in text and an entity-type classifier to assign types to entities. Current span detectors rely on extensive manual labeling to guide training. Almost every span detector requires initial training on basic span features followed by adaptation to task-specific features. This process leads to repetitive training of the basic span features among span detectors. Additionally, metric-based entity-type classifiers, such as prototypical networks, typically employ a specific metric that gauges the distance between the query sample and entity-type referents, ultimately assigning the most probable entity type to the query sample. However, these classifiers encounter the sample dependency problem, primarily stemming from the limited samples available for each entity-type referent. To address these challenges, we proposed an improved few-shot NER pipeline. First, we introduce a steppingstone span detector that is pre-trained on open-domain Wikipedia data. It can be used to initialize the pipeline span detector to reduce the repetitive training of basic features. Second, we leverage a large language model (LLM) to set reliable entity-type referents, eliminating reliance on few-shot samples of each type. Our model exhibits superior performance with fewer training steps and human-labeled data compared with baselines, as demonstrated through extensive experiments on various datasets. Particularly in fine-grained few-shot NER settings, our model outperforms strong baselines, including ChatGPT. We will publicly release the code, datasets, LLM outputs, and model checkpoints.

show abstract

Section: Related Workmentioning

confidence: 99%

Fighting Against the Repetitive Training and Sample Dependency Problem in Few-Shot Named Entity Recognition

Tian,

Yin,

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Prototype networks are the dominant direction for FSRE tasks due to their efficiency. The following are some of the more representative models in FSRE using the prototype network approach: Proto-HATT [17] improves model robustness and accelerates model convergence by designing instance-level and featurelevel attention schemes; MLMAN [18] encodes the instances in the query set and support set and leverages the interaction of local and instance-level information between them to obtain richer semantic representation; REGRAB [44] incorporates global relation graphs with Bayesian meta-learning into the model; TD-proto [45] introduces relation and entity descriptions to enhance prototype networks; ConceptFERE [23] incorporates an entity multiconcept selection module to enhance key entity concept features; CTEG [46] finetuning by assigning pseudolabels to untagged data using the domain; HCRP [20] introduce three modules for hybrid prototype learning, contrast learning, and adaptive focus loss to improve the model; SimpleFSRE [21] simplifies the model using a direct join method of relation information; PRM [22] proposes a parameter-free prototype correction method; and MapRE [47] adds a framework that takes into account label uncertainty and labelaware semantic mapping information. However, these models have inferior domain adaptation capabilities.…”

Section: Few-shot Relation Extractionmentioning

confidence: 99%

“…All [17][18][19] try to fully use the valuable information in the dataset to achieve a more informative prototype network. Recent FSRE research efforts introduce external information such as relation information (description of relation information) [20][21][22] and entity concepts [23], thus effectively improving the performance of FSRE tasks. However, two problems remain: (1) The current work is overly dependent on external information and does not pay sufficient attention to the information already available in the dataset [24].…”

Section: Introductionmentioning

confidence: 99%

FREDA: Few-Shot Relation Extraction Based on Data Augmentation

Liu

Qin

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

The primary task of few-shot relation extraction is to quickly learn the features of relation classes from a few labelled instances and predict the semantic relations between entity pairs in new instances. Most existing few-shot relation extraction methods do not fully utilize the relation information features in sentences, resulting in difficulties in improving the performance of relation classification. Some researchers have attempted to incorporate external information, but the results have been unsatisfactory when applied to different domains. In this paper, we propose a method that utilizes triple information for data augmentation, which can alleviate the issue of insufficient instances and possesses strong domain adaptation capabilities. Firstly, we extract relation and entity pairs from the instances in the support set, forming relation triple information. Next, the sentence information and relation triple information are encoded using the same sentence encoder. Then, we construct an interactive attention module to enable the query set instances to interact separately with the support set instances and relation triple instances. The module pays greater attention to highly interactive parts between instances and assigns them higher weights. Finally, we merge the interacted support set representation and relation triple representation. To our knowledge, we are the first to propose a method that utilizes triple information for data augmentation in relation extraction. In our experiments on the standard datasets FewRel1.0 and FewRel2.0 (domain adaptation), we observed substantial improvements without including external information.

show abstract

“…SimpleFSRE [18] improves model performance by concatenating two representations of relationship information and directly incorporating them into the prototype representation. PRM [19] combines a gating mechanism to utilize relationship description information, determining the degree of preservation and an update of both the prototype and relationship information. CBPM [20] corrects the prototype network by utilizing category information from the query set and hierarchical information from relationship synonyms.…”

Section: Introductionmentioning

confidence: 99%

Multi-Head Self-Attention-Enhanced Prototype Network with Contrastive–Center Loss for Few-Shot Relation Extraction

Ma,

Cheng,

Chen

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

Few-shot relation extraction (FSRE) constitutes a critical task in natural language processing (NLP), involving learning relationship characteristics from limited instances to enable the accurate classification of new relations. The existing research primarily concentrates on using prototype networks for FSRE and enhancing their performance by incorporating external knowledge. However, these methods disregard the potential interactions among different prototype networks, and each prototype network can only learn and infer from its limited instances, which may limit the robustness and reliability of the prototype representations. To tackle the concerns outlined above, this paper introduces a novel prototype network called SACT (multi-head self-attention and contrastive-center loss), aimed at obtaining more comprehensive and precise interaction information from other prototype networks to bolster the reliability of the prototype network. Firstly, SACT employs a multi-head self-attention mechanism for capturing interaction information among different prototypes from traditional prototype networks, reducing the noise introduced by unknown categories with a small sample through information aggregation. Furthermore, SACT introduces a new loss function, the contrastive–center loss function, aimed at tightly clustering samples from a similar relationship category in the center of the feature space while dispersing samples from different relationship categories. Through extensive experiments on FSRE datasets, this paper demonstrates the outstanding performance of SACT, providing strong evidence for the effectiveness and practicality of SACT.

show abstract

Learn from Relation Information: Towards Prototype Representation Rectification for Few-Shot Relation Extraction

Cited by 7 publications

References 15 publications

Fighting Against the Repetitive Training and Sample Dependency Problem in Few-Shot Named Entity Recognition

Fighting Against the Repetitive Training and Sample Dependency Problem in Few-Shot Named Entity Recognition

FREDA: Few-Shot Relation Extraction Based on Data Augmentation

Multi-Head Self-Attention-Enhanced Prototype Network with Contrastive–Center Loss for Few-Shot Relation Extraction

Contact Info

Product

Resources

About