A Neural Pipeline Approach for the PharmaCoNER Shared Task using Contextual Exhaustive Models

Abstract

We present a neural pipeline approach that performs named entity recognition (NER) and concept indexing (CI), which links them to concept unique identifiers (CUIs) in a knowledge base, for the PharmaCoNER shared task on pharmaceutical drugs and chemical entities. We proposed a neural NER model that captures the surrounding semantic information of a given sequence by capturing the forwardand backward-context of bidirectional LSTM (Bi-LSTM) output of a target span using contextual span representation-based exhaustive approach. The NER model enumerates all possible spans as potential entity mentions and classify them into entity types or no entity with deep neural networks. For representing span, we compare several different neural network architectures and their ensembling for the NER model. We then perform dictionary matching for CI and, if there is no matching, we further compute similarity scores between a mention and CUIs using entity embeddings to assign the CUI with the highest score to the mention. We evaluate our approach on the two sub-tasks in the shared task. Among the five submitted runs, the best run for each sub-task achieved the F-score of 86.76% on Sub-task 1 (NER) and the F-score of 79.97% (strict) on Sub-task 2 (CI).

Span-based Named Entity Recognition by Generating and Compressing Information

“…The span representation was calculated based on a pre-trained language model and then classified to a corresponding entity type by a linear layer. Following the suite, numerous studies have shown that span-based approaches to NER could produce SOTA performance (Zheng et al, 2019;Tan et al, 2020;Xia et al, 2019;Sohrab et al, 2019;Xu et al, 2021;Fu et al, 2021;Li et al, 2021;Yu et al, 2022). Fu et al (2021) designed SpanNER that learns the representation of a span based on its token representation and the span length embedding.…”