CODER: Knowledge-infused cross-lingual medical term embedding for term normalization

Yuan, Zheng; Zhao, Zhengyun; Sun, Haixia; Li, Jiao; Wang, Fei; Yu, Sheng

doi:10.1016/j.jbi.2021.103983

Cited by 61 publications

(58 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, in the task to measure the degree of contextual relatedness and similarity between biomedical and clinical terms, they showed decreased performance (Table 2). Furthermore, according to our findings on BERT models, CODER Yuan et al (2022) has better performance, probably because it encodes most of the words without splitting them into their sub-words, as shown in Table 4.…”

Section: Discussionmentioning

confidence: 73%

“…MedTCS outperformed BERTbased models by a significant margin in terms of correlation scores on the UMNSRS-Similarity dataset (Table 2). Moreover, in (Table 3), we have compared our best achieved results with recently reported scores of UMNSRS datasets (Mao and Fung, 2020;Singh and Jin, 2020;Yuan et al, 2022). MedTCS achieved significantly better coverage and correlation scores.…”

Section: Intrinsic Evaluationmentioning

confidence: 92%

See 1 more Smart Citation

Medical terminology-based computing system: a lightweight post-processing solution for out-of-vocabulary multi-word terms

Saeed

Naveed

2022

Front. Mol. Biosci.

View full text Add to dashboard Cite

The linguistic rules of medical terminology assist in gaining acquaintance with rare/complex clinical and biomedical terms. The medical language follows a Greek and Latin-inspired nomenclature. This nomenclature aids the stakeholders in simplifying the medical terms and gaining semantic familiarity. However, natural language processing models misrepresent rare and complex biomedical words. In this study, we present MedTCS—a lightweight, post-processing module—to simplify hybridized or compound terms into regular words using medical nomenclature. MedTCS enabled the word-based embedding models to achieve 100% coverage and enabled the BiowordVec model to achieve high correlation scores (0.641 and 0.603 in UMNSRS similarity and relatedness datasets, respectively) that significantly surpass the n-gram and sub-word approaches of FastText and BERT. In the downstream task of named entity recognition (NER), MedTCS enabled the latest clinical embedding model of FastText-OA-All-300d to improve the F1-score from 0.45 to 0.80 on the BC5CDR corpus and from 0.59 to 0.81 on the NCBI-Disease corpus, respectively. Similarly, in the drug indication classification task, our model was able to increase the coverage by 9% and the F1-score by 1%. Our results indicate that incorporating a medical terminology-based module provides distinctive contextual clues to enhance vocabulary as a post-processing step on pre-trained embeddings. We demonstrate that the proposed module enables the word embedding models to generate vectors of out-of-vocabulary words effectively. We expect that our study can be a stepping stone for the use of biomedical knowledge-driven resources in NLP.

show abstract

Section: Discussionmentioning

confidence: 73%

Section: Intrinsic Evaluationmentioning

confidence: 92%

Medical terminology-based computing system: a lightweight post-processing solution for out-of-vocabulary multi-word terms

Saeed

Naveed

2022

Front. Mol. Biosci.

View full text Add to dashboard Cite

show abstract

“…We want to leverage synonyms knowledge from KB to enhance the model's performance. Injecting synonyms knowledge to the encoder-only models can be done by contrastive learning (Liu et al, 2021a;Yuan et al, 2022). However, such a paradigm cannot directly apply to encoder-decoder architecture as entities are not represented by dense embeddings.…”

Section: Kb-guided Pre-trainingmentioning

confidence: 99%

“…Recent methods in biomedical EL mainly used neural networks to encode mentions and each concept name into the same dense space, then linked mentions to corresponding concepts depending on embedding similarities (Sung et al, 2020;Lai et al, 2021;Bhowmik et al, 2021;Ujiie et al, 2021;Agarwal et al, 2021). Synonyms knowledge has been injected into these similarity-based methods by contrastive learning (Liu et al, 2021a;Yuan et al, 2022). For example in UMLS, concept C0085435 has synonyms: Reiter syndrome, Reactive arthritis and ReA which help models to learn different names of a concept entity.…”

Section: Introductionmentioning

confidence: 99%

Generative Biomedical Entity Linking via Knowledge Base-Guided Pre-training and Synonyms-Aware Fine-tuning

Yuan¹,

Zheng²,

Yu³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Entities lie in the heart of biomedical natural language understanding, and the biomedical entity linking (EL) task remains challenging due to the fine-grained and diversiform concept names. Generative methods achieve remarkable performances in general domain EL with less memory usage while requiring expensive pre-training. Previous biomedical EL methods leverage synonyms from knowledge bases (KB) which is not trivial to inject into a generative method. In this work, we use a generative approach to model biomedical EL and propose to inject synonyms knowledge in it. We propose KBguided pre-training by constructing synthetic samples with synonyms and definitions from KB and require the model to recover concept names. We also propose synonyms-aware fine-tuning to select concept names for training, and propose decoder prompt and multisynonyms constrained prefix tree for inference. Our method achieves state-of-the-art results on several biomedical EL tasks without candidate selection which displays the effectiveness of proposed pre-training and fine-tuning strategies. The source code is available at Github.com/Yuanhy1997/GenBioEL.

show abstract

“…Entity linking task has labeled entities in texts, while ICD coding only provides texts. Synonyms have also been used in biomedical entity linking (Sung et al, 2020;Yuan et al, 2022). BioSYN (Sung et al, 2020) uses marginalization to sum the probabilities of all synonyms as the similarity between a term and a concept.…”

Section: Memory Complexitymentioning

confidence: 99%

Code Synonyms Do Matter: Multiple Synonyms Matching Network for Automatic ICD Coding

Zheng¹,

Tan²,

Huang³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Automatic ICD coding is defined as assigning disease codes to electronic medical records (EMRs). Existing methods usually apply label attention with code representations to match related text snippets. Unlike these works that model the label with the code hierarchy or description, we argue that the code synonyms can provide more comprehensive knowledge based on the observation that the code expressions in EMRs vary from their descriptions in ICD. By aligning codes to concepts in UMLS, we collect synonyms of every code. Then, we propose a multiple synonyms matching network to leverage synonyms for better code representation learning, and finally help the code classification. Experiments on the MIMIC-III dataset show that our proposed method outperforms previous state-of-the-art methods. * Work done at Alibaba DAMO Academy. 1 who.int/standards/classifications/ classification-of-diseases 2 "Label" equals to "code" in some contexts of this paper. 3 Our codes and model can be found at https:// github.com/GanjinZero/ICD-MSMN.

show abstract

CODER: Knowledge-infused cross-lingual medical term embedding for term normalization

Cited by 61 publications

References 34 publications

Medical terminology-based computing system: a lightweight post-processing solution for out-of-vocabulary multi-word terms

Medical terminology-based computing system: a lightweight post-processing solution for out-of-vocabulary multi-word terms

Generative Biomedical Entity Linking via Knowledge Base-Guided Pre-training and Synonyms-Aware Fine-tuning

Code Synonyms Do Matter: Multiple Synonyms Matching Network for Automatic ICD Coding

Contact Info

Product

Resources

About