Low-Resource Named Entity Recognition via the Pre-Training Model

Abstract

Named entity recognition (NER) is an important task in the processing of natural language, which needs to determine entity boundaries and classify them into pre-defined categories. For low-resource languages, most state-of-the-art systems require tens of thousands of annotated sentences to obtain high performance. However, there is minimal annotated data available about Uyghur and Hungarian (UH languages) NER tasks. There are also specificities in each task—differences in words and word order across languages make it a challenging problem. In this paper, we present an effective solution to providing a meaningful and easy-to-use feature extractor for named entity recognition tasks: fine-tuning the pre-trained language model. Therefore, we propose a fine-tuning method for a low-resource language model, which constructs a fine-tuning dataset through data augmentation; then the dataset of a high-resource language is added; and finally the cross-language pre-trained model is fine-tuned on this dataset. In addition, we propose an attention-based fine-tuning strategy that uses symmetry to better select relevant semantic and syntactic information from pre-trained language models and apply these symmetry features to name entity recognition tasks. We evaluated our approach on Uyghur and Hungarian datasets, which showed wonderful performance compared to some strong baselines. We close with an overview of the available resources for named entity recognition and some of the open research questions.

Named entity recognition and emotional viewpoint monitoring in online news using artificial intelligence

“…Currently, due to the increasing influence of pre-training models such as the Bidirectional Encoder Representations from Transformers (BERT) in natural language processing (NLP) research, another study ( Chen et al, 2021 ) introduced pre-training models in NER research to enhance NER models through the powerful semantic representation ability of pre-training models for semantic understanding of text, thus achieving better entity recognition results.…”

Zero-Shot Transfer Learning using Affix and Correlated Cross-Lingual Embeddings.

“…The advent of cross-lingual models has been at the aid of many downstream tasks plagued with low-resourcedness such as document classification 48 , POS tagging 20 , sentiment analyses 49 , and named entity recognition 50 . The primary goal of these models is to learn transferable language-generic knowledge encoded inside sound embedding spaces(of high-resourced languages) obtained from large-enough language representation and inject this mined "knowledge" into the embedding spaces of low-resourced languages to use for downstream tasks.…”

Deep Learning Transformer Architecture for Named Entity Recognition on Low Resourced Languages: State of the art results

“…These IE applications include sequence tagging tasks such as Named-Entity Recognition (NER) and Part-of-Speech (POS) tagging. N NER is a task that processes natural language, classifying and grouping, for example, words into categories (also known as phrase types) [20]. With the advent of big data and large datasets, classifying natural language in these datasets has become increasingly important.…”

“…tions are able to apply NER in customer support, content classification, and search and recommendation engines [21]. Furthermore, NER findings may be transferred to other NLP tasks such as MT, automatic text summarization, and knowledge base construction [20]. Lack of data severely impedes performance on NER tasks with low-resourced languages [20].…”

“…Furthermore, NER findings may be transferred to other NLP tasks such as MT, automatic text summarization, and knowledge base construction [20]. Lack of data severely impedes performance on NER tasks with low-resourced languages [20].…”