BioELECTRA:Pretrained Biomedical text Encoder using Discriminators

Kanakarajan, Kamal raj; Kundumani, Bhuvana; Sankarasubbu, Malaikannan

doi:10.18653/v1/2021.bionlp-1.16

Cited by 53 publications

(33 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…advise, int, effect and mechanism) of the DDI, ChemProt [ 10 ] annotated 5 categories of the chemical–protein interaction and DrugProt [ 40 ], an extension of ChemProt, annotated 13 categories. Recently, ChemProt and DDI13 are widely used in evaluating the abilities of biomedical pretrained language models [ 46–49 ] on RE tasks.…”

Section: Overviews Of Ner/nel/re Datasetsmentioning

confidence: 99%

BioRED: a rich biomedical relation extraction dataset

Luo

Lai

Wei

et al. 2022

Briefings in Bioinformatics

View full text Add to dashboard Cite

Automated relation extraction (RE) from biomedical literature is critical for many downstream text mining applications in both research and real-world settings. However, most existing benchmarking datasets for biomedical RE only focus on relations of a single type (e.g. protein–protein interactions) at the sentence level, greatly limiting the development of RE systems in biomedicine. In this work, we first review commonly used named entity recognition (NER) and RE datasets. Then, we present a first-of-its-kind biomedical relation extraction dataset (BioRED) with multiple entity types (e.g. gene/protein, disease, chemical) and relation pairs (e.g. gene–disease; chemical–chemical) at the document level, on a set of 600 PubMed abstracts. Furthermore, we label each relation as describing either a novel finding or previously known background knowledge, enabling automated algorithms to differentiate between novel and background information. We assess the utility of BioRED by benchmarking several existing state-of-the-art methods, including Bidirectional Encoder Representations from Transformers (BERT)-based models, on the NER and RE tasks. Our results show that while existing approaches can reach high performance on the NER task (F-score of 89.3%), there is much room for improvement for the RE task, especially when extracting novel relations (F-score of 47.7%). Our experiments also demonstrate that such a rich dataset can successfully facilitate the development of more accurate, efficient and robust RE systems for biomedicine. Availability: The BioRED dataset and annotation guidelines are freely available at https://ftp.ncbi.nlm.nih.gov/pub/lu/BioRED/.

show abstract

Section: Overviews Of Ner/nel/re Datasetsmentioning

confidence: 99%

BioRED: a rich biomedical relation extraction dataset

Luo

Lai

Wei

et al. 2022

Briefings in Bioinformatics

View full text Add to dashboard Cite

show abstract

“…The resulting accuracy by f1-metric, micro-averaged over the four relation classes, is 81.46%, which is comparable to the accuracy other language model-based approcahes [18,20,54] achieve for determining relations between entities extracted from this dataset, the state of the art being 84.05% [20].…”

Section: Estimation Of the Relation Extraction Efficiencymentioning

confidence: 57%

Extraction of the Relations among Significant Pharmacological Entities in Russian-Language Reviews of Internet Users on Medications

Sboev

Selivanov

Moloshnikov

et al. 2022

BDCC

View full text Add to dashboard Cite

Nowadays, the analysis of digital media aimed at prediction of the society’s reaction to particular events and processes is a task of a great significance. Internet sources contain a large amount of meaningful information for a set of domains, such as marketing, author profiling, social situation analysis, healthcare, etc. In the case of healthcare, this information is useful for the pharmacovigilance purposes, including re-profiling of medications. The analysis of the mentioned sources requires the development of automatic natural language processing methods. These methods, in turn, require text datasets with complex annotation including information about named entities and relations between them. As the relevant literature analysis shows, there is a scarcity of datasets in the Russian language with annotated entity relations, and none have existed so far in the medical domain. This paper presents the first Russian-language textual corpus where entities have labels of different contexts within a single text, so that related entities share a common context. therefore this corpus is suitable for the task of belonging to the medical domain. Our second contribution is a method for the automated extraction of entity relations in Russian-language texts using the XLM-RoBERTa language model preliminarily trained on Russian drug review texts. A comparison with other machine learning methods is performed to estimate the efficiency of the proposed method. The method yields state-of-the-art accuracy of extracting the following relationship types: ADR–Drugname, Drugname–Diseasename, Drugname–SourceInfoDrug, Diseasename–Indication. As shown on the presented subcorpus from the Russian Drug Review Corpus, the method developed achieves a mean F1-score of 80.4% (estimated with cross-validation, averaged over the four relationship types). This result is 3.6% higher compared to the existing language model RuBERT, and 21.77% higher compared to basic ML classifiers.

show abstract

“…If the pretraining corpus is more similar to the text of the downstream task, it will have better results. Therefore, we surveyed the related literature and experimental reports to select the following pretrained models: BERTweet ( 21 ), DeBERTa ( 22 ), BioBERT ( 23 ) and BioELECTRA ( 24 ). Table 1 shows the pretraining resources and corpus size of these models.…”

Section: Methodsmentioning

confidence: 99%

Task reformulation and data-centric approach for Twitter medication name extraction

et al. 2022

View full text Add to dashboard Cite

Automatically extracting medication names from tweets is challenging in the real world. There are many tweets; however, only a small proportion mentions medications. Thus, datasets are usually highly imbalanced. Moreover, the length of tweets is very short, which makes it hard to recognize medication names from the limited context. This paper proposes a data-centric approach for extracting medications in the BioCreative VII Track 3 (Automatic Extraction of Medication Names in Tweets). Our approach formulates the sequence labeling problem as text entailment and question–answer tasks. As a result, without using the dictionary and ensemble method, our single model achieved a Strict F1 of 0.77 (the official baseline system is 0.758, and the average performance of participants is 0.696). Moreover, combining the dictionary filtering and ensemble method achieved a Strict F1 of 0.804 and had the highest performance for all participants. Furthermore, domain-specific and task-specific pretrained language models, as well as data-centric approaches, are proposed for further improvements. Database URL https://competitions.codalab.org/competitions/23925 and https://biocreative.bioinformatics.udel.edu/tasks/biocreative-vii/track-3/

show abstract

BioELECTRA:Pretrained Biomedical text Encoder using Discriminators

Cited by 53 publications

References 34 publications

BioRED: a rich biomedical relation extraction dataset

BioRED: a rich biomedical relation extraction dataset

Extraction of the Relations among Significant Pharmacological Entities in Russian-Language Reviews of Internet Users on Medications

Task reformulation and data-centric approach for Twitter medication name extraction

Contact Info

Product

Resources

About