Effective Feature Representation for Clinical Text Concept Extraction

Tao, Yifeng; Godefroy, Bruno; Genthial, Guillaume; Potts, Christopher

doi:10.18653/v1/w19-1901

Cited by 10 publications

(4 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Here, the wikipages were restricted to those that contained SNOMED‐CT concepts. They evaluated the embeddings with three Drug‐Disease Link Prediction datasets as provided by Godefroy and Potts (2019), Dingwall and Potts (2018) and Tao et al (2019). While retrofitting, in both settings, improved the embeddings' performance.…”

Section: Discussionmentioning

confidence: 99%

Word embeddings for biomedical natural language processing: A survey

Chiu

Baker

2020

Language and Linguist. Compass

View full text Add to dashboard Cite

Word representations are mathematical objects that capture the semantic and syntactic properties of words in a way that is interpretable by machines. Recently, encoding word properties into low‐dimensional vector spaces using neural networks has become increasingly popular. Word embeddings are now used as the main input to natural language processing (NLP) applications, achieving cutting‐edge results. Nevertheless, most word‐embedding studies are carried out with general‐domain text and evaluation datasets, and their results do not necessarily apply to text from other domains (e.g., biomedicine) that are linguistically distinct from general English. To achieve maximum benefit when using word embeddings for biomedical NLP tasks, they need to be induced and evaluated using in‐domain resources. Thus, it is essential to create a detailed review of biomedical embeddings that can be used as a reference for researchers to train in‐domain models. In this paper, we review biomedical word embedding studies from three key aspects: the corpora, models and evaluation methods. We first describe the characteristics of various biomedical corpora, and then compare popular embedding models. After that, we discuss different evaluation methods for biomedical embeddings. For each aspect, we summarize the various challenges discussed in the literature. Finally, we conclude the paper by proposing future directions that will help advance research into biomedical embeddings.

show abstract

Section: Discussionmentioning

confidence: 99%

Word embeddings for biomedical natural language processing: A survey

Chiu

Baker

2020

Language and Linguist. Compass

View full text Add to dashboard Cite

show abstract

“…Outcome prediction from clinical text has been studied regarding a variety of outcomes. The most prevalent being in-hospital mortality (Ghassemi et al, 2014;Jo et al, 2017;Suresh et al, 2018;Si and Roberts, 2019), diagnosis prediction (Tao et al, 2019;Liu et al, 2018Liu et al, , 2019a and phenotyping (Liu et al, 2019b;Jain et al, 2019;Oleynik et al, 2019;Pfaff et al, 2020). In recent years, most approaches are based on deep neural networks due to their ability to outperform earlier methods in most settings.…”

Section: Clinical Outcome Predictionmentioning

confidence: 99%

What Do You See in this Patient? Behavioral Testing of Clinical NLP Models

Aken¹,

Herrmann²,

Löser³

2022

Proceedings of the 4th Clinical Natural Language Processing Workshop

View full text Add to dashboard Cite

Decision support systems based on clinical notes have the potential to improve patient care by pointing doctors towards overseen risks. Predicting a patient's outcome is an essential part of such systems, for which the use of deep neural networks has shown promising results. However, the patterns learned by these networks are mostly opaque and previous work revealed both reproduction of systemic biases and unexpected behavior for out-of-distribution patients. For application in clinical practice it is crucial to be aware of such behavior. We thus introduce a testing framework that evaluates clinical models regarding certain changes in the input. The framework helps to understand learned patterns and their influence on model decisions. In this work, we apply it to analyse the change in behavior with regard to the patient characteristics gender, age and ethnicity. Our evaluation of three current clinical NLP models demonstrates the concrete effects of these characteristics on the models' decisions. They show that model behavior varies drastically even when fine-tuned on the same data with similar AUROC score. These results exemplify the need for a broader communication of model behavior in the clinical domain.

show abstract

“…While gene embeddings can be directly learned using the GIT model, it has been shown in the field of NLP that the pre-trained word embeddings can significantly improve the performance in other related NLP tasks. 12,14 Such pre-trained word embeddings can capture the knowledge of co-occurrence pattern of the words in languages and exhibit sound semantic properties: words of similar semantic meanings are close in embedding space, e.g., e "each" ≈ e "every" . We therefore propose an algorithm called "Gene2Vec" to pre-train the gene embeddings, which is closely related the skip gram word2vec 12 pre-training algorithm.…”

Section: S2 Gene2vec Algorithm Implementationmentioning

confidence: 99%

“…This representation simply indicates which gene is perturbed, but it does not reflect the functional impact of the SGA, nor can it represent the similarity of distinct SGAs that perturb a common signaling pathway. We conjecture that it is possible to represent an SGA as a low-dimensional vector, in the same manner as the "word embedding" [12][13][14] in the natural language processing (NLP) field, such that the representation reflects the functional impact of a gene on biological systems, and genes sharing similar functions should be closely located in such embedding space. Here the "similar function" is broadly defined, e.g., genes from the same pathway or of the same biological process.…”

Section: Introductionmentioning

confidence: 99%

From genome to phenome: Predicting multiple cancer phenotypes based on somatic genomic alterations via the genomic impact transformer

et al. 2019

Self Cite

View full text Add to dashboard Cite

Cancers are mainly caused by somatic genomic alterations (SGAs) that perturb cellular signaling systems and eventually activate oncogenic processes. Therefore, understanding the functional impact of SGAs is a fundamental task in cancer biology and precision oncology. Here, we present a deep neural network model with encoder-decoder architecture, referred to as genomic impact transformer (GIT), to infer the functional impact of SGAs on cellular signaling systems through modeling the statistical relationships between SGA events and differentially expressed genes (DEGs) in tumors. The model utilizes a multi-head self-attention mechanism to identify SGAs that likely cause DEGs, or in other words, differentiating potential driver SGAs from passenger ones in a tumor. GIT model learns a vector (gene embedding) as an abstract representation of functional impact for each SGA-affected gene. Given SGAs of a tumor, the model can instantiate the states of the hidden layer, providing an abstract representation (tumor embedding) reflecting characteristics of perturbed molecular/cellular processes in the tumor, which in turn can be used to predict multiple phenotypes. We apply the GIT model to 4,468 tumors profiled by The Cancer Genome Atlas (TCGA) project. The attention mechanism enables the model to better capture the statistical relationship between SGAs and DEGs than conventional methods, and distinguishes cancer drivers from passengers. The learned gene embeddings capture the functional similarity of SGAs perturbing common pathways. The tumor embeddings are shown to be useful for tumor status representation, and phenotype prediction including patient survival time and drug response of cancer cell lines. *

show abstract

Effective Feature Representation for Clinical Text Concept Extraction

Cited by 10 publications

References 29 publications

Word embeddings for biomedical natural language processing: A survey

Word embeddings for biomedical natural language processing: A survey

What Do You See in this Patient? Behavioral Testing of Clinical NLP Models

From genome to phenome: Predicting multiple cancer phenotypes based on somatic genomic alterations via the genomic impact transformer

Contact Info

Product

Resources

About