A Boundary-aware Neural Model for Nested Named Entity Recognition

Abstract

In natural language processing, it is common that many entities contain other entities inside them. Most existing works on named entity recognition (NER) only deal with flat entities but ignore nested ones. We propose a boundary-aware neural model for nested NER which leverages entity boundaries to predict entity categorical labels. Our model can locate entities precisely by detecting boundaries using sequence labeling models. Based on the detected boundaries, our model utilizes the boundary-relevant regions to predict entity categorical labels, which can decrease computation cost and relieve error propagation problem in layered sequence labeling model. We introduce multitask learning to capture the dependencies of entity boundaries and their categorical labels, which helps to improve the performance of identifying entities. We conduct our experiments on nested NER datasets and the experimental results demonstrate that our model outperforms other state-of-the-art methods.

A Bidirectional Iterative Algorithm for Nested Named Entity Recognition

“…Each layer has a CRF output generating predictions for the current level of nesting, and its hidden states are passed to the next layer until no new entities are detected. Zheng et al [25] have trained an LSTM-based multitask model to jointly detect the boundaries of named entities and classify them. Recent research has started to utilize transformer [26] architecture.…”

“…More details can be found in the PolEval subsection. [11] 74.5 66.0 70.0 Finkel and Manning [12] 75.4 65.9 70.3 Lu and Roth [13] 74.2 66.7 70.3 Muis and Lu [14] 75.4 66.8 70.8 Wang and Lu [15] 76.2 67.5 71.6 Neural methods Xu et al [17] 71.2 64.3 67.6 Katiyar and Cardie [16] 79.8 68.2 73.6 Ju et al [24] 78.5 71.3 74.7 Wang et al [22] 78.0 70.2 73.9 Wang and Lu [15] 77.0 73.3 75.1 Sohrab and Miwa [18] 93.2 64.0 77.1 Marinho et al [23] 74.0 72.0 73.0 Lin et al [19] 75.8 73.9 74.8 Zheng et al [25] 75.9 73.6 74.7 Sun et al [27] 77.4 74.9 76.2 Shibuya and Hovy [28] 78 To make a fair comparison with other publications, we preprocessed the corpus following the guidance of Finkel and Manning [12]. Their procedure, which involved splitting the dataset and reducing the number of entity types, was reused by most of the studies included in our comparison.…”

“…In this experiment, we replaced the language model and word embeddings trained on the MEDLINE/PubMed corpus with original character-level LM by Akbik et al [35] trained on the news corpus, and 300-dimensional GloVe embeddings by Pennington et al [46]. To the best of our knowledge, Sohrab and Miwa [18], Zheng et al [25], Sun et al [27] and Shibuya and Hovy [28] used biomedical word representations in their work. Without domain pre-training, our algorithm performs 1.77% worse in terms of F1-score.…”

“…Baseline (c) is a concatenation of predictions from the first two models. In addition, we have retrained and performed an evaluation of other recent methods, implementations of which are publicly available: Zheng et al [25], Ju et al [24], and Shibuya and Hovy [28]. We can see that the linear classifier policy is again the best performing one, achieving an F1-score of 94.1%, an improvement of 0.6% compared to the work of Shibuya and Hovy [28], and a significant increase over all other methods.…”

“…Our method can handle a set of deeply nested entities that were undetectable with simpler models. [18] 75.0 60.8 67.2 Ju et al [24] 72.9 61.5 66.7 Zheng et al [25] 74…”