Large Margin Prototypical Network for Few-shot Relation Classification with Fine-grained Features

Fan, Miao; Bai, Yinjuan; Sun, Mingming; Li, Ping

doi:10.1145/3357384.3358100

Cited by 29 publications

(17 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To overcome the catastrophic forgetting problem, Cai et al [20] introduced a two-phase prototypical network, which adapted prototype attention alignment and triplet loss to dynamically recognize the novel relations with a few support instances without catastrophic forgetting. Similarly, Fan et al [21] proposed the large-margin prototypical network with fine-grained features (LM-ProtoNet), which could generalize well on few-shot relations classification. To learn predictive and robust relation representations from the training phase, Ding et al [22] proposed prototype learning methods with geometric interpretation, where the prototypes were unit vectors uniformly dispersed in a unit ball, and the sentence embeddings were centered at the end of their corresponding prototype vectors.…”

Section: Few-shot Relation Classificationmentioning

confidence: 99%

Commonsense Knowledge-Aware Prompt Tuning for Few-Shot NOTA Relation Classification

Jin

Zhang

et al. 2022

Applied Sciences

View full text Add to dashboard Cite

Compared with the traditional few-shot task, the few-shot none-of-the-above (NOTA) relation classification focuses on the realistic scenario of few-shot learning, in which a test instance might not belong to any of the target categories. This undoubtedly increases the task’s difficulty because given only a few support samples, this cannot represent the distribution of NOTA categories in space. The model needs to make full use of the syntactic information and word meaning information learned in the pre-training stage to distinguish the NOTA category and the support sample category in the embedding space. However, previous fine-tuning methods mainly focus on optimizing the extra classifiers (on top of pre-trained language models (PLMs)) and neglect the connection between pre-training objectives and downstream tasks. In this paper, we propose the commonsense knowledge-aware prompt tuning (CKPT) method for a few-shot NOTA relation classification task. First, a simple and effective prompt-learning method is developed by constructing relation-oriented templates, which can further stimulate the rich knowledge distributed in PLMs to better serve downstream tasks. Second, external knowledge is incorporated into the model by a label-extension operation, which forms knowledgeable prompt tuning to improve and stabilize prompt tuning. Third, to distinguish the NOTA pairs and positive pairs in embedding space more accurately, a learned scoring strategy is proposed, which introduces a learned threshold classification function and improves the loss function by adding a new term focused on NOTA identification. Experiments on two widely used benchmarks (FewRel 2.0 and Few-shot TACRED) show that our method is a simple and effective framework, and a new state of the art is established in the few-shot classification field.

show abstract

Section: Few-shot Relation Classificationmentioning

confidence: 99%

Commonsense Knowledge-Aware Prompt Tuning for Few-Shot NOTA Relation Classification

Jin

Zhang

et al. 2022

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Additionally, a large number of contrastive losses such as triplet loss [38] and large margin nearest neighbor (LMNN) [39] are proposed to maximize the similarity between positive pairs while minimizing those of the negatives. Inspired by these works, contrastive loss such as [40], [41] are leveraged to learn a more discriminant embedding space in episodic training for metric-learning-based few-shot classification. The most related work to the present study is the additive margin loss proposed in [40], which directly adds a margin on the distance between different classes.…”

Section: Related Workmentioning

confidence: 99%

“…WLMNC loss adopted in our work shares a similar idea to the classrelevant additive margin loss; however, it is implemented by the weighted hinge loss and does not leverage any additional semantic information. Another relevant work is the triplet loss applied to prototypical networks in [41], which attempts to maintain a large margin between the distances from a given prototype of the support set to a same-labeled query example and to a differently labeled query example. This triplet loss is different from our proposed LMNC (WLMNC) loss, which aims to maintain a large margin between the distances from a given query example to its target prototype and to differently labeled prototypes.…”

Section: Related Workmentioning

confidence: 99%

Enhancing Metric-Based Few-Shot Classification With Weighted Large Margin Nearest Center Loss

2021

View full text Add to dashboard Cite

Metric-learning-based methods, which attempt to learn a deep embedding space on extremely large episodes, have been successfully applied to few-shot classification problems. In this paper, we propose the adoption of large margin nearest center (LMNC) loss during episodic training to enhance metriclearning-based few-shot classification methods. Loss functions (such as cross-entropy and mean square error) commonly used in episodic training strive to achieve the strict goal that differently labeled examples in the embedding space are separated by an infinite distance. However, the learned embedding space cannot guarantee that this goal will be achieved for every episode sampled from a large number of classes. Instead of an infinite distance, LMNC loss requires only that differently labeled examples be separated by a large margin, which can well relax the strict constraint of the traditional loss functions, easily leading to a discriminative embedding space. Moreover, considering the multilevel similarity between various classes, we alleviate the constraint of a fixed large margin and extend LMNC loss to weighted LMNC (WLMNC) loss, which can effectively take advantage of interclass information, achieving a more separable embedding space with adaptive interclass margins. Experiments on state-of-the-art benchmarks demonstrate that the adoption of LMNC and WLMNC losses can strongly improve the embedding learning performance and classification accuracy of metric-based few-shot classification methods for various few-shot scenarios. In particular, LMNC and WLMNC losses can obtain 1.86% and 2.46% gains in prototypical network on miniImageNet for 5-way 1-shot scenario, respectively.INDEX TERMS few-shot classification, metric learning, large margin nearest center loss, weighted large margin nearest center loss

show abstract

“…From the example shown in Figure 2, we can see that, in the new scenario of distantly supervised FSRC, both support and query sets are practically noisy. If a single false positive instance is sampled as a query like previous studies (Han et al, 2018;Fan et al, 2019;Gao et al, 2019), it cannot be classified into an appropriate relation in the support set. Since a few-shot model is optimized by minimizing the loss of the predictions over the queries, sampling a mislabeled instance as the query will inevitably mislead the optimization process.…”

Section: Motivationmentioning

confidence: 99%

“…The query set Q gives rise to the major difference with respect to the formulation of conventional FSRC in Han et al (2018), Fan et al (2019), andGao et al (2019). As DS data tends to have mislabeled instances, if the previous formulation is followed, a mislabeled instance is likely to be sampled as the query instance; in this case, a FSRC model may be intermittently confused during training by the mislabeled query instances, as they substantially deviate from real ones.…”

Section: Task Formulationmentioning

confidence: 99%

Boosting Knowledge Base Automatically via Few-Shot Relation Classification

Pang

Tan

et al. 2020

Front. Neurorobot.

View full text Add to dashboard Cite

Relation classification (RC) aims at extracting structural information, i.e., triplets of two entities with a relation, from free texts, which is pivotal for automatic knowledge base construction. In this paper, we investigate a fully automatic method to train a RC model which facilitates to boost the knowledge base. Traditional RC models cannot extract new relations unseen during training since they define RC as a multiclass classification problem. The recent development of few-shot learning (FSL) provides a feasible way to accommodate to fresh relation types with a handful of examples. However, it requires a moderately large amount of training data to learn a promising few-shot RC model, which consumes expensive human labor. This issue recalls a kind of weak supervision methods, dubbed distant supervision (DS), which can generate the training data automatically. To this end, we propose to investigate the task of few-shot relation classification under distant supervision. As DS naturally brings in mislabeled training instances, to alleviate the negative impact, we incorporate various multiple instance learning methods into the classic prototypical networks, which can achieve sentence-level noise reduction. In experiments, we evaluate our proposed model under the standard N-way K-shot setting of few-shot learning. The experiment results show that our proposal achieves better performance.

show abstract

Large Margin Prototypical Network for Few-shot Relation Classification with Fine-grained Features

Cited by 29 publications

References 14 publications

Commonsense Knowledge-Aware Prompt Tuning for Few-Shot NOTA Relation Classification

Commonsense Knowledge-Aware Prompt Tuning for Few-Shot NOTA Relation Classification

Enhancing Metric-Based Few-Shot Classification With Weighted Large Margin Nearest Center Loss

Boosting Knowledge Base Automatically via Few-Shot Relation Classification

Contact Info

Product

Resources

About