Long short-term memory RNN for biomedical named entity recognition

Abstract

BackgroundBiomedical named entity recognition(BNER) is a crucial initial step of information extraction in biomedical domain. The task is typically modeled as a sequence labeling problem. Various machine learning algorithms, such as Conditional Random Fields (CRFs), have been successfully used for this task. However, these state-of-the-art BNER systems largely depend on hand-crafted features.ResultsWe present a recurrent neural network (RNN) framework based on word embeddings and character representation. On top of the neural network architecture, we use a CRF layer to jointly decode labels for the whole sentence. In our approach, contextual information from both directions and long-range dependencies in the sequence, which is useful for this task, can be well modeled by bidirectional variation and long short-term memory (LSTM) unit, respectively. Although our models use word embeddings and character embeddings as the only features, the bidirectional LSTM-RNN (BLSTM-RNN) model achieves state-of-the-art performance — 86.55% F1 on BioCreative II gene mention (GM) corpus and 73.79% F1 on JNLPBA 2004 corpus.ConclusionsOur neural network architecture can be successfully used for BNER without any manual feature engineering. Experimental results show that domain-specific pre-trained word embeddings and character-level representation can improve the performance of the LSTM-RNN models. On the GM corpus, we achieve comparable performance compared with other systems using complex hand-crafted features. Considering the JNLPBA corpus, our model achieves the best results, outperforming the previously top performing systems. The source code of our method is freely available under GPL at https://github.com/lvchen1989/BNER.

Introducing Information Extraction to Radiology Information Systems to Improve the Efficiency on Reading Reports

“…Lyu et al introduces a variation of recurrent neural network (RNN) called bidirectional long short-term memory units (BLSTM) to recognize biomedical-named entities. 20 Lakhani et al uses convolutional neural network (CNN) to extracting ICD-O-3 topographic codes from pathology reports. 7 Gao et al uses a hierarchical attention networks for information extraction from pathology reports.…”

Biomedical Concept Recognition Using Deep Neural Sequence Models

“…Korvigo et al [26] applied a CNN-RNN network to recognize spans of chemicals and Luo et al 2018 [28] proposed attention-based bidirectional LSTM with CRF to detect spans of chemicals. Unanue et al, 2017 [29] used bidirectional LSTM with CRF to detect spans of drug names and clinical concepts, while Lyu et al 2017 [27] proposed bidirectional LSTM-RNN model for detecting spans of a variety of biomedical concepts. However, none of these approaches also attempted the normalization step, so they did not identify which particular concept in an ontology was detected.…”

Comparing CNN and LSTM character-level embeddings in BiLSTM-CRF models for chemical and disease named entity recognition

“…Features that capture word-internal characteristics have been shown to be effective for BNER tasks in CRF models (Klinger et al, 2008). Lyu et al (2017) applied a BiLSTM-CRF model with LSTM-based character-level word embeddings to a gene and protein NER task, demonstrating state-of-art performance that outperformed traditional feature-based models. Luo et al (2018) further improved on this result on a chemical NER task by adding an attention layer between the BiL-STM and CRF layers (Att-BiLSTM-CRF).…”