In this work, we study the problem of named entity recognition (NER) in a low resource scenario, focusing on few-shot and zero-shot settings. Built upon large-scale pre-trained language models, we propose a novel NER framework, namely SpanNER, which learns from natural language supervision and enables the identification of never-seen entity classes without using in-domain labeled data. We perform extensive experiments on 5 benchmark datasets and evaluate the proposed method in the few-shot learning, domain transfer and zero-shot learning settings. The experimental results show that the proposed method can bring 10%, 23% and 26% improvements in average over the best baselines in few-shot learning, domain transfer and zero-shot learning settings respectively. Dataset Example ErrorCoNLL03 Japan then laid siege to the Syrian penalty area for most of the game but rarely breached the Syrian defence. Gold: Japan → LOC Gold: Syrian → MISC Gold: Syrian → MISC CoNLL03 Japan then laid siege to the Syrian penalty area for most of the game but rarely breached the Syrian defence. Pred: Syrian → MISC Pred: Syrian → MISC CoNLL03 In sentencing Darrel Voeks , 38, to a 10-year prison term on Thursday, Outagmie County Circuit Court Judge Dennis Luebke said he was "a thief by habit." Gold: Darrel Voeks → PER Gold: Outagmie County → LOC Gold: Dennis Luebke → PER CoNLL03 In sentencing Darrel Voeks , 38 , to a 10-year prison term on Thursday, Outagmie County Circuit Court Judge Dennis Luebke said he was "a thief by habit." Pred: Darrel Voeks → PER Pred: 38 → MISC Pred: Outagmie County Circuit Court → ORG Pred: Dennis Luebke → PER CoNLL03 You are narcissitic, Luebke said at the sentencing, adding Voeks should pay restitution of more than $100,000 to the farming family who had hired him. Gold: Voeks → PER CoNLL03 You are narcissitic, Luebke said at the sentencing, adding Voeks should pay restitution of more than $100,000 to the farming family who had hired him.
In this work, we study the problem of named entity recognition (NER) in a low resource scenario, focusing on few-shot and zero-shot settings. Built upon large-scale pre-trained language models, we propose a novel NER framework, namely SpanNER, which learns from natural language supervision and enables the identification of never-seen entity classes without using in-domain labeled data. We perform extensive experiments on 5 benchmark datasets and evaluate the proposed method in the few-shot learning, domain transfer and zero-shot learning settings. The experimental results show that the proposed method can bring 10%, 23% and 26% improvements in average over the best baselines in few-shot learning, domain transfer and zero-shot learning settings respectively.
Background: Real world data analysis problems often require nonlinear methods to get successful prediction. Kernel methods, e.g. Kernelized Principal Component Analysis, are a common way to get nonlinear properties based on linear representations in a high-dimensional feature space. Unfortunately, traditional kernel methods are unscalable for large-size or even medium-size data. On the other hand, randomized algorithms have been recently proposed to extract nonlinear features in kernel methods. Compared with exact kernel methods, this family of approaches is capable of speeding up the training process dramatically, while maintaining acceptable the classification accuracy. However, these methods fail to engage discriminative features. This significantly limits their classification accuracy. Results: In this paper, we propose a scalable and approximate technique called SDRNF for introducing both nonlinear and discriminative features based on randomized methods. By combining randomized kernel approximation with a couple of generalized eigenvector problems, the proposed approach proves both scalable and accurate for large-scale data. Conclusion: A series of experiments on two benchmark data sets MNIST and CIFAR-10 reveal that our method is fast and scalable, and also generates better classification accuracy over other competitive kernel approximation methods.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.