A Korean named entity recognition method using Bi-LSTM-CRF and masked self-attention

Jin, Guozhe; Yu, Zhezhou

doi:10.1016/j.csl.2020.101134

Cited by 20 publications

(7 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…LSTM is essentially derived from the recurrent neural network (RNN) [19]. RNN is an extraordinary network for processing serial data.…”

Section: Long Short-term Memory (Lstm) Networkmentioning

confidence: 99%

Legal Text Recognition Using LSTM-CRF Deep Learning Model

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

In legal texts, named entity recognition (NER) is researched using deep learning models. First, the bidirectional (Bi)-long short-term memory (LSTM)-conditional random field (CRF) model for studying NER in legal texts is established. Second, different annotation methods are used to compare and analyze the entity recognition effect of the Bi-LSTM-CRF model. Finally, other objective loss functions are set to compare and analyze the entity recognition effect of the Bi-LSTM-CRF model. The research results show that the F1 value of the model trained on the word sequence labeling corpus on the named entity is 88.13%, higher than that of the word sequence labeling corpus. For the two types of entities, place names and organization names, the F1 values obtained by the Bi-LSTM-CRF model using word segmentation are 67.60% and 89.45%, respectively, higher than the F1 values obtained by the model using character segmentation. Therefore, the Bi-LSTM-CRF model using word segmentation is more suitable for recognizing extended entities. The parameter learning result using log-likelihood is better than that using the maximum interval criterion, and it is ideal for the Bi-LSTM-CRF model. This method provides ideas for the research of legal text recognition and has a particular value.

show abstract

“…LSTM is essentially derived from the recurrent neural network (RNN) [19]. RNN is an extraordinary network for processing serial data.…”

Section: Long Short-term Memory (Lstm) Networkmentioning

confidence: 99%

Legal Text Recognition Using LSTM-CRF Deep Learning Model

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

show abstract

“…Zhang et al [42] merged pre-trained Glove and word2vec vectors. Although most of the research studies in NER used the pre-trained word embeddings as features, fixing the values of the vectors during model training, there exist some studies that fine-tuned these pre-trained embeddings during the training process [43]- [44]. To reduce the impact of Out of Vocabulary (OOV) words, word level representations are concatenated with character level representations.…”

Section: Related Workmentioning

confidence: 99%

Classical Arabic Named Entity Recognition Using Variant Deep Neural Network Architectures and BERT

Alsaaran

Alrabiah

2021

IEEE Access

View full text Add to dashboard Cite

Recurrent Neural Networks (RNNs) and transformers are deep learning models that have achieved remarkable success in several Natural Language Processing (NLP) tasks since they do not rely on handcrafted features nor enormous knowledge resources. Named Entity Recognition (NER) is an essential NLP task that is used in many applications such as information retrieval, question answering, and machine translation. NER aims to locate, extract, and classify named entities into predefined categories such as person, organization and location. Arabic NER is considered a challenging task because of the complexity and the unique characteristics of Arabic. Most of the previous research on deep learning based-Arabic NER focused on Modern Standard Arabic and Dialectal Arabic, which are different variations from Classical Arabic. In this paper, we investigate deep learning-based Classical Arabic NER using different deep neural network architectures and a BERT based contextual language model that is trained on general domain Arabic text. We propose two RNN-based models by fine-tunning the pretrained BERT language model to recognize and classify named entities from Classical Arabic. The pre-trained BERT contextual language model representations were used as input features to a BGRU/BLSTM model and were fine-tuned using a Classical Arabic NER dataset. In addition, we explore variant architectures of the proposed BERT-BGRU/BLSTM-CRF models. Experimentations showed that the BERT-BGRU-CRF model outperformed the other models by achieving an F-measure of 94.76% on the CANERCorpus. To the best of our knowledge, this is the first work that aims to recognize named entities in Classical Arabic using deep learning.

show abstract

“…The first is the softmax function direct tag prediction which is straightforward to compute. The second is CRF, a well-known statistical graphical model which has demonstrated state-of-the-art accuracy on many sequence labeling tasks Jin and Yu, 2021). The third is a variant of RNN (Goller and Kuchler, 1996) (e.g.…”

Section: Sequence Labeling Modelsmentioning

confidence: 99%

Boundary Detection with BERT for Span-level Emotion Cause Analysis

Li¹,

Gao²,

Feng³

et al. 2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

View full text Add to dashboard Cite

Emotion cause analysis (ECA) has been an emerging topic in natural language processing, which aims to identify the reasons behind a certain emotion expressed in the text. Most ECA methods intend to identify the clause which contains the cause of a given emotion, but such clause-level ECA (CECA) can be ambiguous and imprecise. In this paper, we aim at span-level ECA (SECA) by detecting the precise boundaries of text spans conveying accurate emotion causes from the given context. We formulate this task as sequence labeling and position identification problems and design two neural methods to solve them. Experiments on two benchmark ECA datasets show that the proposed methods substantially outperform the existing ECA models 1 . 1 The code and datasets can be found at https://github.com/xxxyyy2020/boundary-master 2 Bold: Gold emotion cause span; Underline: emotion expression. [xi] represents the i-th clause. The original report is in Chinese and translated into English.

show abstract

A Korean named entity recognition method using Bi-LSTM-CRF and masked self-attention

Cited by 20 publications

References 5 publications

Legal Text Recognition Using LSTM-CRF Deep Learning Model

Legal Text Recognition Using LSTM-CRF Deep Learning Model

Classical Arabic Named Entity Recognition Using Variant Deep Neural Network Architectures and BERT

Boundary Detection with BERT for Span-level Emotion Cause Analysis

Contact Info

Product

Resources

About