A BERT-Based Hybrid System for Chemical Identification and Indexing in Full-Text Articles

Erdengasileng, Arslan; Li, Keqiao; Han, Qing; Tian, Shubo; Wang, Jian; Hu, Ting; Zhang, Jinfeng

doi:10.1101/2021.10.27.466183

Cited by 4 publications

(2 citation statements)

References 13 publications

(12 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The first technique is inspired by Erdengasileng et al. ( 86 ), where the authors propose to randomly replace chemical entities and nonchemical tokens with random strings. The intuition to replace a chemical entity with a random string is to force the model to explore the context when predicting an entity as chemical, since the only way to correctly predict that random string as an chemical entity is with the help of the context.…”

Section: Methodsmentioning

confidence: 99%

Chemical identification and indexing in PubMed full-text articles using deep learning and heuristics

et al. 2022

View full text Add to dashboard Cite

The identification of chemicals in articles has attracted a large interest in the biomedical scientific community, given its importance in drug development research. Most of previous research have focused on PubMed abstracts, and further investigation using full-text documents is required because these contain additional valuable information that must be explored. The manual expert task of indexing Medical Subject Headings (MeSH) terms to these articles later helps researchers find the most relevant publications for their ongoing work. The BioCreative VII NLM-Chem track fostered the development of systems for chemical identification and indexing in PubMed full-text articles. Chemical identification consisted in identifying the chemical mentions and linking these to unique MeSH identifiers. This manuscript describes our participation system and the post-challenge improvements we made. We propose a three-stage pipeline that individually performs chemical mention detection, entity normalization and indexing. Regarding chemical identification, we adopted a deep-learning solution that utilizes the PubMedBERT contextualized embeddings followed by a multilayer perceptron and a conditional random field tagging layer. For the normalization approach, we use a sieve-based dictionary filtering followed by a deep-learning similarity search strategy. Finally, for the indexing we developed rules for identifying the more relevant MeSH codes for each article. During the challenge, our system obtained the best official results in the normalization and indexing tasks despite the lower performance in the chemical mention recognition task. In a post-contest phase we boosted our results by improving our named entity recognition model with additional techniques. The final system achieved 0.8731, 0.8275 and 0.4849 in the chemical identification, normalization and indexing tasks, respectively. The code to reproduce our experiments and run the pipeline is publicly available. Database URL https://github.com/bioinformatics-ua/biocreativeVII_track2

show abstract

Section: Methodsmentioning

confidence: 99%

Chemical identification and indexing in PubMed full-text articles using deep learning and heuristics

et al. 2022

View full text Add to dashboard Cite

show abstract

“…In addition to the above strategies, we also tried some task-specific methods. In this paper, we will focus on Track 2, for which we have performed very well overall ( 36 ).…”

Section: Methodsmentioning

confidence: 99%

Pre-trained models, data augmentation, and ensemble learning for biomedical information extraction and document classification

et al. 2022

Self Cite

View full text Add to dashboard Cite

Large volumes of publications are being produced in biomedical sciences nowadays with ever-increasing speed. To deal with the large amount of unstructured text data, effective natural language processing (NLP) methods need to be developed for various tasks such as document classification and information extraction. BioCreative Challenge was established to evaluate the effectiveness of information extraction methods in biomedical domain and facilitate their development as a community-wide effort. In this paper, we summarize our work and what we have learned from the latest round, BioCreative Challenge VII, where we participated in all five tracks. Overall, we found three key components for achieving high performance across a variety of NLP tasks: (1) pre-trained NLP models; (2) data augmentation strategies and (3) ensemble modelling. These three strategies need to be tailored towards the specific tasks at hands to achieve high-performing baseline models, which are usually good enough for practical applications. When further combined with task-specific methods, additional improvements (usually rather small) can be achieved, which might be critical for winning competitions. Database URL: https://doi.org/10.1093/database/baac066

show abstract

Processing of clinical notes for efficient diagnosis with feedback attention–based BiLSTM

Kolukula,

Puli,

Babi

et al. 2024

Med Biol Eng Comput

View full text Add to dashboard Cite

A BERT-Based Hybrid System for Chemical Identification and Indexing in Full-Text Articles

Cited by 4 publications

References 13 publications

Chemical identification and indexing in PubMed full-text articles using deep learning and heuristics

Chemical identification and indexing in PubMed full-text articles using deep learning and heuristics

Pre-trained models, data augmentation, and ensemble learning for biomedical information extraction and document classification

Processing of clinical notes for efficient diagnosis with feedback attention–based BiLSTM

Contact Info

Product

Resources

About