The CHEMDNER corpus of chemicals and drugs and its annotation principles

Abstract

The automatic extraction of chemical information from text requires the recognition of chemical entity mentions as one of its key steps. When developing supervised named entity recognition (NER) systems, the availability of a large, manually annotated text corpus is desirable. Furthermore, large corpora permit the robust evaluation and comparison of different approaches that detect chemicals in documents. We present the CHEMDNER corpus, a collection of 10,000 PubMed abstracts that contain a total of 84,355 chemical entity mentions labeled manually by expert chemistry literature curators, following annotation guidelines specifically defined for this task. The abstracts of the CHEMDNER corpus were selected to be representative for all major chemical disciplines. Each of the chemical entity mentions was manually labeled according to its structure-associated chemical entity mention (SACEM) class: abbreviation, family, formula, identifier, multiple, systematic and trivial. The difficulty and consistency of tagging chemicals in text was measured using an agreement study between annotators, obtaining a percentage agreement of 91. For a subset of the CHEMDNER corpus (the test set of 3,000 abstracts) we provide not only the Gold Standard manual annotations, but also mentions automatically detected by the 26 teams that participated in the BioCreative IV CHEMDNER chemical mention recognition task. In addition, we release the CHEMDNER silver standard corpus of automatically extracted mentions from 17,000 randomly selected PubMed abstracts. A version of the CHEMDNER corpus in the BioC format has been generated as well. We propose a standard for required minimum information about entity annotations for the construction of domain specific corpora on chemical and drug entities. The CHEMDNER corpus and annotation guidelines are available at: http://www.biocreative.org/resources/biocreative-iv/chemdner-corpus/

Empowering citizens to manage their chemical exposures step 1 ‐ identify ingredients in consumer products

“…For this reason, several efforts have been underway to provide standardized corpora for testing. For example, the CHEMDNER corpus contains 10 K abstracts that have been manually annotated with trivial chemical names (30.36%) systematic names (22.69%), chemical abbreviations (15.55%), chemical formulas (14.26%) and chemical families (14.15%), along with chemical identifiers (2.16%) and text that captures more than one type of chemical entity (0.70%) (Krallinger et al, ). Table 1 summarizes how well some systems identify the entities in the CHEMDNER corpus.…”

Putting hands to rest: efficient deep CNN-RNN architecture for chemical named entity recognition with no hand-crafted rules

“…To facilitate the development of new and superior NER systems, BioCreative announced the CHEMDNER challenge, which ended in 2015 [1]. As part of this task, a team of experts has produced an extensive manually annotated corpus covering various chemical entity types, including systematic and trivial names, abbreviations and identifiers, formulae and phrases.…”

“…As part of this task, a team of experts has produced an extensive manually annotated corpus covering various chemical entity types, including systematic and trivial names, abbreviations and identifiers, formulae and phrases. Due to many difficulties inherent to chemical entity detection and normalisation [1], even manual annotation yields the interannotator agreement score of 91%, which can be regarded as the theoretical limit for any automatic system trained on this corpus. Twenty six teams have submitted their NER systems for the challenge, best of which have reached the F1 score of ∼ 72 − 88% [2,3,4,5,6,7,8,9] on two subtasks: chemical entity mention (CEM) and chemical document indexing (CDI).…”

Web Apps Come of Age for Molecular Sciences

“…), recognition and processing of proper names including persons and places, acronyms, numbers, etc., verb conjugation detection and change, negation, single/plural detection and conversion, stemming and normalization procedures, etc. Both come with prebuilt lexicons, i.e., collections of words "understood" by the mining algorithm, which are rather generic but can be extended as required for specific applications by using lexicons built for specific disciplines [58][59][60]. For applications in chemistry and biology, lexicons could in turn be extended by using tools for conversion between molecular formulas, structures and names, annotations and ontologies from databases, possibly even on-the-fly through JavaScript API calls to external web services or by mining knowledge databases such as DBpedia (the structured form mirror of Wikipedia) or Wordnik (a meta-dictionary).…”