Time-sensitive clinical concept embeddings learned from large electronic health records

Xiang, Yang; Xu, Jun; Si, Yuqi; Li, Zhiheng; Rasmy, Laila; Zhou, Yujia; Tiryaki, Firat; Li, Fang; Zhang, Yaoyun; Wu, Yonghui; Jiang, Xiaoqian; Zheng, W. Jim; Zhi, Degui; Tao, Cui; Xu, Hua

doi:10.1186/s12911-019-0766-3

Cited by 22 publications

(26 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our work validates their findings that their embeddings offer the best performance. However, it would be interesting to also consider the recent clinical concept embeddings developed by (Xiang et al, 2019). They use a similar amount of data (50 million) as Beam et al, using a large dataset from electronic health records, and apply a novel method to incorporate time-sensitive information.…”

Section: Discussion and Future Directionmentioning

confidence: 99%

“…and available for multiple medical fields. For instance, the recent work by Xiang et al (2019) compared embeddings trained by different methodologies on a task predicting the onset of heart failure (Rasmy et al, 2018). This would be an appropri-ate task to judge embeddings from "Diseases of the Circulatory System"; others would be needed for other systems.…”

Section: Discussion and Future Directionmentioning

confidence: 99%

See 1 more Smart Citation

Comparing the Intrinsic Performance of Clinical Concept Embeddings by Their Field of Medicine

Nunez¹,

Carenini

2019

Proceedings of the Tenth International Workshop on Health Text Mining and Information Analysis (LOUHI 2019)

View full text Add to dashboard Cite

Pre-trained word embeddings are becoming increasingly popular for natural languageprocessing tasks. This includes medical applications, where embeddings are trained for clinical concepts using specific medical data. Recent work continues to improve on these embeddings. However, no one has yet sought to determine whether these embeddings work as well for one field of medicine as they do in others. In this work, we use intrinsic methods to evaluate embeddings from the various fields of medicine as defined by their ICD-9 systems. We find significant differences between fields, and motivate future work to investigate whether extrinsic tasks will follow a similar pattern.

show abstract

Section: Discussion and Future Directionmentioning

confidence: 99%

Section: Discussion and Future Directionmentioning

confidence: 99%

Comparing the Intrinsic Performance of Clinical Concept Embeddings by Their Field of Medicine

Nunez¹,

Carenini

2019

Proceedings of the Tenth International Workshop on Health Text Mining and Information Analysis (LOUHI 2019)

View full text Add to dashboard Cite

show abstract

“…This adds a layer of complexity, as modelers must choose if and how to incorporate time into their models. 7 , 8 …”

Section: Introductionmentioning

confidence: 99%

“…This sparsity of data would typically increase the value of techniques such as transfer learning, which allow modelers to “transfer” concepts learned from one dataset to another; however, surprisingly few pretrained concept embeddings have been published. 7 Those that have been published have typically followed the ICD-9 standard, which has been discontinued for several years. 6 , 7 …”

Section: Introductionmentioning

confidence: 99%

Exploiting hierarchy in medical concept embedding*

et al. 2021

View full text Add to dashboard Cite

Objective To construct and publicly release a set of medical concept embeddings for codes following the ICD-10 coding standard which explicitly incorporate hierarchical information from medical codes into the embedding formulation. Materials and Methods We trained concept embeddings using several new extensions to the Word2Vec algorithm using a dataset of approximately 600,000 patients from a major integrated healthcare organization in the Mid-Atlantic US. Our concept embeddings included additional entities to account for the medical categories assigned to codes by the Clinical Classification Software Revised (CCSR) dataset. We compare these results to sets of publicly released pretrained embeddings and alternative training methodologies. Results We found that Word2Vec models which included hierarchical data outperformed ordinary Word2Vec alternatives on tasks which compared naïve clusters to canonical ones provided by CCSR. Our Skip-Gram model with both codes and categories achieved 61.4% normalized mutual information with canonical labels in comparison to 57.5% with traditional Skip-Gram. In models operating on two different outcomes, we found that including hierarchical embedding data improved classification performance 96.2% of the time. When controlling for all other variables, we found that co-training embeddings improved classification performance 66.7% of the time. We found that all models outperformed our competitive benchmarks. Discussion We found significant evidence that our proposed algorithms can express the hierarchical structure of medical codes more fully than ordinary Word2Vec models, and that this improvement carries forward into classification tasks. As part of this publication, we have released several sets of pretrained medical concept embeddings using the ICD-10 standard which significantly outperform other well-known pretrained vectors on our tested outcomes.

show abstract

“…Similarly, the above extrinsic evaluation of heart-failure prediction is also based on a patient's clinical notes [8]. Developing embeddings for biological concepts and applications is also important.…”

Section: Introductionmentioning

confidence: 99%

BioConceptVec: Creating and evaluating literature-based biomedical concept embeddings on a large scale

et al. 2020

View full text Add to dashboard Cite

A massive number of biological entities, such as genes and mutations, are mentioned in the biomedical literature. The capturing of the semantic relatedness of biological entities is vital to many biological applications, such as protein-protein interaction prediction and literature-based discovery. Concept embeddings-which involve the learning of vector representations of concepts using machine learning models-have been employed to capture the semantics of concepts. To develop concept embeddings, named-entity recognition (NER) tools are first used to identify and normalize concepts from the literature, and then different machine learning models are used to train the embeddings. Despite multiple attempts, existing biomedical concept embeddings generally suffer from suboptimal NER tools, small-scale evaluation, and limited availability.In response, we employed high-performance machine learning-based NER tools for concept recognition and trained our concept embeddings, BioConceptVec, via four different machine learning models on ~30 million PubMed abstracts. BioConceptVec covers over 400,000 biomedical concepts mentioned in the literature and is of the largest among the publicly available biomedical concept embeddings to date. To evaluate the validity and utility of BioConceptVec, we respectively performed two intrinsic evaluations (identifying related concepts based on drug-gene and gene-gene interactions) and two extrinsic evaluations (protein-protein interaction prediction and drug-drug interaction extraction), collectively using over 25 million instances from nine independent datasets (17 million instances from six intrinsic evaluation tasks and 8 million instances from three extrinsic evaluation tasks), which is, by far, the most comprehensive to our best knowledge. The intrinsic evaluation results demonstrate that BioConceptVec consistently has, by a large margin, better performance than existing concept embeddings in identifying similar and related concepts. More importantly, the extrinsic evaluation results demonstrate that using BioConceptVec with advanced deep learning models can significantly improve performance in downstream bioinformatics studies and biomedical text-mining applications.Our BioConceptVec embeddings and benchmarking datasets are publicly available at https://github.com/ncbi-nlp/BioConceptVec. Author SummaryCapturing the semantics of related biological concepts, such as genes and mutations, is of significant importance to many research tasks in computational biology such as protein-protein interaction detection, gene-drug association prediction, and biomedical literature-based discovery. Here, we propose to leverage state-of-the-art text mining tools and machine learning models to learn the semantics via vector representations (aka. embeddings) of over 400,000 biological concepts mentioned in the entire PubMed abstracts. Our learned embeddings, namely BioConceptVec, can capture related concepts based on their surrounding contextual information in the literature, which is beyond exact term ...

show abstract

Time-sensitive clinical concept embeddings learned from large electronic health records

Cited by 22 publications

References 18 publications

Comparing the Intrinsic Performance of Clinical Concept Embeddings by Their Field of Medicine

Comparing the Intrinsic Performance of Clinical Concept Embeddings by Their Field of Medicine

Exploiting hierarchy in medical concept embedding*

BioConceptVec: Creating and evaluating literature-based biomedical concept embeddings on a large scale

Contact Info

Product

Resources

About