Biomedical Named-Entity Recognition by Hierarchically Fusing BioBERT Representations and Deep Contextual-Level Word-Embedding

Abstract

Text mining in the biomedical domain is increasingly important as the volume of biomedical documents increases. Thanks to advances in natural language processing (NLP), extracting valuable information from the biomedical literature is gaining popularity among researchers, and deep learning has enabled the development of effective biomedical text mining models. However, directly applying advancements in NLP to biomedical sources often yields unsatisfactory results, due to a word distribution drift from the general language domain corpora to specific biomedical corpora, and this drift introduces linguistic ambiguities. To overcome these challenges, this paper presents a novel method for biomedical named entity-recognition (BioNER) through hierarchically fusing representations from BioBERT, which is trained on biomedical corpora and Deep contextual-level word embeddings to handle the linguistic challenges within biomedical literature. Proposed text representation is then fed to attention-based Bi-directional Long Short Term Memory (BiLSTM) with Conditional random field (CRF) for the BioNER task. The experimental analysis shows that our proposed end-to-end methodology outperforms existing state-ofthe-art methods for the BioNER task.

COVIDSenti: A Large-Scale Benchmark Twitter Data Set for COVID-19 Sentiment Analysis

“…Over the years, text analysis has been used in various applications: email filtering [10] , irony and sarcasm detection [33] , document organization [19] , sentiment and opinion mining prediction [30] , [37] , hate speech detection [29] , [35] , question answering systems [18] , content mining [1] , biomedical text mining [31] , [32] , and more. Twitter data have seen wide usage for emotional analysis [3] , [9] , [41] .…”

SentiMedQAer: A Transfer Learning-Based Sentiment-Aware Model for Biomedical Question Answering

“…BioBERT (Lee et al, 2020 ) is a BERT variant pre-trained on PubMed articles for adapting the biomedical domain. There are two typical ways to apply BioBERT to downstream tasks: first, BioBERT can be fine-tuned on a specific dataset to suit the target learning task (Jin et al, 2019 ); second, BioBERT can be treated as a neural encoder that transforms word tokens of input texts to word embeddings (Naseem et al, 2020 ). Since BioBERT has been pre-trained on a large biomedical corpus with over a million PubMed articles, it presents superior performance in a variety of biomedical NLP tasks, compared to BERT and other pre-training models (Lee et al, 2020 ).…”

A Comparative Analysis of Active Learning for Biomedical Text Mining

“…NLP is the intersection of computing science and linguistics that includes dissecting and understanding common human language from both speech and written texts. Over the years, NLP has been used in various applications such as email filtering [36], irony and sarcasm detection [37] document organisation [38], sentiment and opinion mining prediction [39][40][41], hate speech detection [42][43][44], question answering [45], content mining [46], biomedical text mining [47,48], and many more [8,49,50].…”

“…Along these lines, standard NLP ad- 47 vances and frameworks cannot be straightforwardly applied to the clinical domain [5]. 48 ML-based algorithms, rule-based and existing dictionary-based methods can be uti- 49 lised to identify and extract the concepts from raw text corpus in finance, medical, and 50 various other domains [6][7][8][9][10]. In the clinical domain, the ShARe/CLEF 2013 eHealth Eval- 51 uation Lab and the i2b2/VA challenge methodologies have been applied in shared tasks 52 [11][12][13].…”