Simple and Effective Few-Shot Named Entity Recognition with Structured Nearest Neighbor Learning

Yang, Yi; Katiyar, Arzoo

doi:10.18653/v1/2020.emnlp-main.516

Cited by 116 publications

(155 citation statements)

References 12 publications

Supporting

Mentioning

113

Contrasting

Order By: Relevance

“…In order to consider label dependencies that are essential in slot tagging tasks (Huang et al, 2015), Hou et al (2020) proposed a collapsed dependency transfer (CDT) mechanism by simulating transition scores for the target domain from transition probabilities among BIO labels in the source domain, outperforming previous methods on slot filling by a large margin. Yang and Katiyar (2020) further explored the transition probability by evenly distributing the collapsed transition scores to the target domain to maintain a valid distribution. However, this simulation is noisy and the difference between the source and target domains can result in biased transition probabilities.…”

Section: Metric Learning In Language Understandingmentioning

confidence: 99%

Few-shot Intent Classification and Slot Filling with Retrieved Examples

Yu¹,

He²,

Zhang³

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

Few-shot learning arises in important practical scenarios, such as when a natural language understanding system needs to learn new semantic labels for an emerging, resource-scarce domain. In this paper, we explore retrieval-based methods for intent classification and slot filling tasks in few-shot settings. Retrieval-based methods make predictions based on labeled examples in the retrieval index that are similar to the input, and thus can adapt to new domains simply by changing the index without having to retrain the model. However, it is nontrivial to apply such methods on tasks with a complex label space like slot filling. To this end, we propose a span-level retrieval method that learns similar contextualized representations for spans with the same label via a novel batch-softmax objective. At inference time, we use the labels of the retrieved spans to construct the final structure with the highest aggregated score. Our method outperforms previous systems in various few-shot settings on the CLINC and SNIPS benchmarks.

show abstract

Section: Metric Learning In Language Understandingmentioning

confidence: 99%

Few-shot Intent Classification and Slot Filling with Retrieved Examples

Yu¹,

He²,

Zhang³

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

show abstract

“…Few-shot Sequence Labeling In recent years, several works (Fritzler et al, 2019;Hou et al, 2020;Yang and Katiyar, 2020) have been proposed to solve the few-shot named entity recognition using sequence labeling methods. Fritzler et al (2019) applied the vanilla CRF in the few-shot scenario directly.…”

Section: Related Workmentioning

confidence: 99%

“…Hou et al (2020) proposed a collapsed dependency transfer mechanism (CDT) into CRF, which learns label dependency patterns of a set of task-agnostic abstract labels and utilizes these patterns as transition scores for novel labels. Yang and Katiyar (2020) trains their model on the training data in a standard supervised learning manner and then uses the prototypical networks and the CDT for prediction in the inference phase. Different from these methods learning the transition scores by optimization, we build a network to generate the transition scores based on the label prototypes instead.…”

Section: Related Workmentioning

confidence: 99%

“…2) CDT (Hou et al, 2020): As the SOTA of the few-shot NER task, we reimplement it according to the official code and adapt it in the FSED task to replace our Transition Module. 3) StructShot (Yang and Katiyar, 2020): It is also a few-shot NER model. It first pre-trains on the training set and utilizes the prototypical networks and the CDT for prediction based on the support set in the inference phase.…”

Section: Paradigmmentioning

confidence: 99%

See 1 more Smart Citation

Few-Shot Event Detection with Prototypical Amortized Conditional Random Field

Cong¹,

Cui²,

Yu³

et al. 2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

View full text Add to dashboard Cite

Event detection tends to struggle when it needs to recognize novel event types with a few samples. The previous work attempts to solve this problem in the identify-then-classify manner but ignores the trigger discrepancy between event types, thus suffering from the error propagation. In this paper, we present a novel unified model which converts the task to a few-shot tagging problem with a double-part tagging scheme. To this end, we first propose the Prototypical Amortized Conditional Random Field (PA-CRF) to model the label dependency in the few-shot scenario, which approximates the transition scores between labels based on the label prototypes. Then Gaussian distribution is introduced for modeling of the transition scores to alleviate the uncertain estimation resulting from insufficient data. Experimental results show that the unified models work better than existing identifythen-classify models and our PA-CRF further achieves the best results on the benchmark dataset FewEvent. Our code and data are available at http://github.com/congxin95/ PA-CRF. * Corresponding Author Marry It served as the location of Bogart's [wedding] to Bacall.

show abstract

“…NER has been addressed in several works. In Yang and Katiyar, 2020) the task of interest consists of recognizing one class of named entities, for tag set extension or domain transfer. In our work, we extend the N-way K-shot setting to structured prediction.…”

Section: Related Workmentioning

confidence: 99%

Proceedings of the 1st Workshop on Meta Learning and Its Applications to Natural Language Processing

2021

View full text Add to dashboard Cite

Deep learning based natural language processing (NLP) has become the mainstream of research in recent years and significantly outperforms conventional methods. However, deep learning models are notorious for being data and computation hungry. These downsides limit such models' application from deployment to different domains, languages, countries, or styles, since collecting in-genre data and model training from scratch are costly. The long-tail nature of human language makes challenges even more significant.Meta-learning, or 'Learning to Learn', aims to learn better learning algorithms, including better parameter initialization, optimization strategy, network architecture, distance metrics, and beyond. Metalearning has been shown to allow faster fine-tuning, converge to better performance, and achieve outstanding results for few-shot learning in many applications. Meta-learning is one of the most important new techniques in machine learning in recent years, but the method is mainly investigated with applications in computer vision. It is believed that meta-learning has excellent potential to be applied in NLP, and some works have been proposed with notable achievements in several relevant problems, e.g., relation extraction, machine translation, and dialogue generation and state tracking. However, it does not catch the same level of attention as in the image processing community.

show abstract

Simple and Effective Few-Shot Named Entity Recognition with Structured Nearest Neighbor Learning

Cited by 116 publications

References 12 publications

Few-shot Intent Classification and Slot Filling with Retrieved Examples

Few-shot Intent Classification and Slot Filling with Retrieved Examples

Few-Shot Event Detection with Prototypical Amortized Conditional Random Field

Proceedings of the 1st Workshop on Meta Learning and Its Applications to Natural Language Processing

Contact Info

Product

Resources

About