CWPC_BiAtt: Character–Word–Position Combined BiLSTM-Attention for Chinese Named Entity Recognition

Abstract

Usually taken as linguistic features by Part-Of-Speech (POS) tagging, Named Entity Recognition (NER) is a major task in Natural Language Processing (NLP). In this paper, we put forward a new comprehensive-embedding, considering three aspects, namely character-embedding, word-embedding, and pos-embedding stitched in the order we give, and thus get their dependencies, based on which we propose a new Character–Word–Position Combined BiLSTM-Attention (CWPC_BiAtt) for the Chinese NER task. Comprehensive-embedding via the Bidirectional Llong Short-Term Memory (BiLSTM) layer can get the connection between the historical and future information, and then employ the attention mechanism to capture the connection between the content of the sentence at the current position and that at any location. Finally, we utilize Conditional Random Field (CRF) to decode the entire tagging sequence. Experiments show that CWPC_BiAtt model we proposed is well qualified for the NER task on Microsoft Research Asia (MSRA) dataset and Weibo NER corpus. A high precision and recall were obtained, which verified the stability of the model. Position-embedding in comprehensive-embedding can compensate for attention-mechanism to provide position information for the disordered sequence, which shows that comprehensive-embedding has completeness. Looking at the entire model, our proposed CWPC_BiAtt has three distinct characteristics: completeness, simplicity, and stability. Our proposed CWPC_BiAtt model achieved the highest F-score, achieving the state-of-the-art performance in the MSRA dataset and Weibo NER corpus.

Named Entity Recognition of Ancient Poems Based on Albert-BiLSTM-MHA-CRF Model

“…In addition, the F1 values of entity recognition of the method proposed in this paper and other methods based on deep learning on public datasets in three different domains are shown in Table 7. These models are lattice LSTM [30], a lstm model that fully considers word and character information; CAN_NER [31], a character-based local attention layer convolutional neural network (CNN) and gated recurrent unit (GRU) with global self-attention layer; CWPC_BiAtt [32], a attention-based bilstm model combining character and word position information; and ACNN [33], a model combining 7, Albert-BiLSTM-MHA-CRF has improved the effect of entity recognition on Weibo and Resume datasets. Compared with Bert-BiLSTM-CRF, the entity recognition effect of Albert-BiLSTM-MHA-CRF is increased by 5.51% and 0.41% on F1 value, respectively.…”

Systematic Literature Review on Named Entity Recognition: Approach, Method, and Application

“…English is the largest language dataset available for NER research because English is the most widely spoken language globally. NER research in other languages include Arabic [64,117,125,132,255], Chinese [51,52,105,156,159,180,256], German [269], Korean [115,130,196,258] and Indian [35,112,119,141]. Among the NER studies in Bahasa Indonesia have been conducted by Wintaka et [284].…”

“…Entities in the farming domain are such pests and diseases [99], geology (e.g., rock, stratum, toponym) [48], sports science (e.g., competition name, level of competition, match time) [103], and military domain (e.g., weapons, mission, location, organization) [104]. However, there are some studies in several fields at once, such as those conducted by Zhong et al [87], Johnson et al [105], Jin et al [106], Ekbal and Saha [107], Xu et al [108],band Song et al [109].…”

Research on Joint Extraction Method of Entity and Relation Triples Based on Hierarchical Cascade Labeling

“…LSTM (Long Short-Term Memory) network is a special form of an RNN (Recurrent Neural Network), which can better handle the sequence data for example the text data [39]. Compared with the traditional RNN model, LSTM can solve the problem of the long gradient disappearance of the text [40].…”