Comparison of natural language processing tools for automatic gene ontology annotation of scientific literature

Abstract

Abstract-Manual curation of scientific literature for ontologybased knowledge representation has proven infeasible and unscalable to the large and growing volume of scientific literature. Automated annotation solutions that leverage text mining and Natural Language Processing (NLP) have been developed to ameliorate the problem of literature curation. These NLP approaches use parsing, syntactical, and lexical analysis of text to recognize and annotate pieces of text with ontology concepts. Here, we conduct a comparison of four state of the art NLP tools at the task of recognizing Gene Ontology concepts from biomedical literature using the Colorado Richly Annotated Full-Text (CRAFT) corpus as a gold standard reference. We demonstrate the use of semantic similarity metrics to compare NLP tool annotations to the gold standard.

Automated ontology-based annotation of scientific literature using deep learning

“…CNNs were also used for biomedical named entity recognition combined with n-gram character embeddings resulting in enhanced performance in a comparison with other deep learning models [25]. A comprehensive review of deep learning methods for named entity recognition can be found in [13] and a comparison of existing text mining tools in [3].…”

Assessment of text coherence using an ontology‐based relatedness measurement method

“…To the best of our knowledge, there is no "gold standard" for the annotation rate. Various studies conducting semantic annotation in various domains, such as Bada, Vasilevsky, Haendel, and Hunter (2016)) and Beasley and Manda (2018)) in bioinformatics, Lévy, Tomeh, and Ma (2014))…”

“…To the best of our knowledge, there is no “gold standard” for the annotation rate. Various studies conducting semantic annotation in various domains, such as Bada, Vasilevsky, Haendel, and Hunter ()) and Beasley and Manda ()) in bioinformatics, Lévy, Tomeh, and Ma ()) and Fiorelli, Pazienza, and Stellato ()) in annotation tool development, Jadidinejad, Mahmoudi, and Meybodi ()) in document classification, and Ali et al () in knowledge engineering; try to maximize the number of words annotated but do not report any number for annotation rate. We report on our findings regarding the annotation rate in Section 5.4.…”

Data mining powered by the gene ontology

“…The large majority of text mining approaches for recognizing ontology concepts from text either rely on lexical and syntactic analysis of text in addition to machine learning (Cui et al, ; Jonquet et al, ; Manda, Beasley, & Mohanty, ; Mungall et al, ). Beasley and Manda () recently conducted a comparison of a number of text mining tools at annotating biological literature with GO terms.…”