A Neural Multi-Task Learning Framework to Jointly Model Medical Named Entity Recognition and Normalization

Zhao, Sendong; Liu, Ting; Zhao, Sicheng; Wang, Fei

doi:10.1609/aaai.v33i01.3301817

Cited by 105 publications

(92 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Lee et al [20] Sachan et al [47] Wang et al [46] Xu et al [49] Hong and Lee [51] Beltagy et al [19] Xu et al [49] Zhao et al [39] Zhao et al [39] Sachan et al [47] Lee et al [20] Hong and Lee [51] Lou et al [48] Wang et al [46] Lou et al [48] Corpus BioBERT v1.1 [20]: self-trained BERT on PubMed pre-trained embeddings + self-trained on PubMed pre-trained: [16] word embeddings [50] self-trained on PubMed and PMC pre-trained word embeddings (PubMed): [16,52] SciBERT [19]: self-trained BERT on biomedical full texts from Semantic Scholar word embeddings [50] self-trained on PubMed and PMC pre-trained: [16,[53][54][55] pre-trained: [16,[53][54][55] pre-trained embeddings + self-trained on PubMed BioBERT V1.1 [20]: self-trained BERT on PubMed pre-trained word embeddings (PubMed): [16,52] n/a pre-trained: [16] n/a…”

Section: Citationsmentioning

confidence: 99%

“…Even more ambitious is the task of linking (or grounding) textual mentions and semantic types to unique identifiers of a given terminology or ontology (such as SNOMED-CT, ICD, or the Human Disease Ontology, https://www.ebi.ac.uk/ ols/ontologies/doid), an issue we will not elaborate on in this survey, cf. e.g., [39]. 7 Concrete numbers in the column "Number of Mentions," indicating the number of named entity mentions (possibly split into training, development, and test set, if provided), may slightly differ for the same corpus because of data cleansing (e.g., removal of duplicates), different pre-processing (e.g., tokenization), and other version issues.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Medical Information Extraction in the Age of Deep Learning

Hahn

Oleynik

2020

Yearb Med Inform

View full text Add to dashboard Cite

Objectives: We survey recent developments in medical Information Extraction (IE) as reported in the literature from the past three years. Our focus is on the fundamental methodological paradigm shift from standard Machine Learning (ML) techniques to Deep Neural Networks (DNNs). We describe applications of this new paradigm concentrating on two basic IE tasks, named entity recognition and relation extraction, for two selected semantic classes—diseases and drugs (or medications)—and relations between them. Methods: For the time period from 2017 to early 2020, we searched for relevant publications from three major scientific communities: medicine and medical informatics, natural language processing, as well as neural networks and artificial intelligence. Results: In the past decade, the field of Natural Language Processing (NLP) has undergone a profound methodological shift from symbolic to distributed representations based on the paradigm of Deep Learning (DL). Meanwhile, this trend is, although with some delay, also reflected in the medical NLP community. In the reporting period, overwhelming experimental evidence has been gathered, as illustrated in this survey for medical IE, that DL-based approaches outperform non-DL ones by often large margins. Still, small-sized and access-limited corpora create intrinsic problems for data-greedy DL as do special linguistic phenomena of medical sublanguages that have to be overcome by adaptive learning strategies. Conclusions: The paradigm shift from (feature-engineered) ML to DNNs changes the fundamental methodological rules of the game for medical NLP. This change is by no means restricted to medical IE but should also deeply influence other areas of medical informatics, either NLP- or non-NLP-based.

show abstract

Section: Citationsmentioning

confidence: 99%

mentioning

confidence: 99%

Medical Information Extraction in the Age of Deep Learning

Hahn

Oleynik

2020

Yearb Med Inform

View full text Add to dashboard Cite

show abstract

“…The subsequent layers are inspired by the work of Zhao et al (2019), who propose a multi-tasklearning framework to jointly tackle span detection (NER) and normalization (NEN). A key step to make NER and NEN compatible was to model NEN as a sequence-labeling problem, where IDs are predicted for each token just like span tags in NER (cf.…”

Section: Bilstm-based Systemmentioning

confidence: 99%

UZH@CRAFT-ST: a Sequence-labeling Approach to Concept Recognition

Furrer¹,

Cornelius²,

Rinaldi³

2019

Proceedings of the 5th Workshop on BioNLP Open Shared Tasks

View full text Add to dashboard Cite

As our submission to the CRAFT shared task 2019, we present two neural approaches to concept recognition. We propose two different systems for joint named entity recognition (NER) and normalization (NEN), both of which model the task as a sequence labeling problem. Our first system is a BiLSTM network with two separate outputs for NER and NEN trained from scratch, whereas the second system is an instance of BioBERT fine-tuned on the concept-recognition task. We exploit two strategies for extending concept coverage, ontology pretraining and backoff with a dictionary lookup. Our results show that the backoff strategy effectively tackles the problem of unseen concepts, addressing a major limitation of the chosen design. In the cross-system comparison, BioBERT proves to be a strong basis for creating a concept-recognition system, although some entity types are predicted more accurately by the BiLSTM-based system.

show abstract

“…In this work, we take the task a step further from existing monolingual research in a single domain [2,3,6,12,13,20,22] by exploring multilingual transfer between EHRs and UGTs in different languages. Our goal is not to outperform state of the art models on each dataset separately, but to ask whether we can transfer knowledge from a high-resource language, such as English, to a lowresource one, e.g., Russian, for NER of biomedical entities.…”

Section: Introductionmentioning

confidence: 99%

On Biomedical Named Entity Recognition: Experiments in Interlingual Transfer for Clinical and Social Media Texts

Miftahutdinov

Alimova

Tutubalina

2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Although deep neural networks yield state-of-the-art performance in biomedical named entity recognition (bioNER), much research shares one limitation: models are usually trained and evaluated on English texts from a single domain. In this work, we present a fine-grained evaluation intended to understand the efficiency of multilingual BERT-based models for bioNER of drug and disease mentions across two domains in two languages, namely clinical data and user-generated texts on drug therapy in English and Russian. We investigate the role of transfer learning (TL) strategies between four corpora to reduce the number of examples that have to be manually annotated. Evaluation results demonstrate that multi-BERT shows the best transfer capabilities in the zero-shot setting when training and test sets are either in the same language or in the same domain. TL reduces the amount of labeled data needed to achieve high performance on three out of four corpora: pretrained models reach 98-99% of the full dataset performance on both types of entities after training on 10-25% of sentences. We demonstrate that pretraining on data with one or both types of transfer can be effective.

show abstract

A Neural Multi-Task Learning Framework to Jointly Model Medical Named Entity Recognition and Normalization

Cited by 105 publications

References 5 publications

Medical Information Extraction in the Age of Deep Learning

Medical Information Extraction in the Age of Deep Learning

UZH@CRAFT-ST: a Sequence-labeling Approach to Concept Recognition

On Biomedical Named Entity Recognition: Experiments in Interlingual Transfer for Clinical and Social Media Texts

Contact Info

Product

Resources

About