Learning Disentangled Representations of Texts with Application to Biomedical Abstracts

Jain, Sarthak; Banner, Edward A.; Meent, Jan-Willem van de; Marshall, Iain J.; Wallace, Byron C.

doi:10.18653/v1/d18-1497

Cited by 19 publications

(17 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Their results suggest that, in absence of a suitable supervision signal, highquality factors can only be learned in the presence of a strong inductive bias. Going beyond unsupervised approaches, (Jain et al, 2018) propose a supervised approach for DRL for text. As supervision signal, they use triplets of the form (s, d, o) a which encode that relative to aspect a, it holds that s and d are more similar than d and o.…”

Section: Disentangled Representation Learning (Drl)mentioning

confidence: 99%

Learning Conceptual Spaces with Disentangled Facets

Alshaikh¹,

Bouraoui²,

Schockaert³

2019

Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL)

View full text Add to dashboard Cite

Conceptual spaces are geometric representations of meaning that were proposed by Gärdenfors (2000). They share many similarities with the vector space embeddings that are commonly used in natural language processing. However, rather than representing entities in a single vector space, conceptual spaces are usually decomposed into several facets, each of which is then modelled as a relatively lowdimensional vector space. Unfortunately, the problem of learning such conceptual spaces has thus far only received limited attention. To address this gap, we analyze how, and to what extent, a given vector space embedding can be decomposed into meaningful facets in an unsupervised fashion. While this problem is highly challenging, we show that useful facets can be discovered by relying on word embeddings to group semantically related features.

show abstract

Section: Disentangled Representation Learning (Drl)mentioning

confidence: 99%

Learning Conceptual Spaces with Disentangled Facets

Alshaikh¹,

Bouraoui²,

Schockaert³

2019

Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL)

View full text Add to dashboard Cite

show abstract

“…Drawing inspiration from the vision community, learning disentangled representations has also been investigated in areas such as natural language processing and graph analysis. (Jain et al, 2018) propose an autoencoders architecture to disentangle the populations, interventions, and outcomes in biomedical texts. propose a prism module for semantic disentanglement in named entity recognition.…”

Section: Learning Disentangled Representationsmentioning

confidence: 99%

Knowledge Router: Learning Disentangled Representations for Knowledge Graphs

Zhang¹,

Rao²,

Tay³

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

The design of expressive representations of entities and relations in a knowledge graph is an important endeavor. While many of the existing approaches have primarily focused on learning from relational patterns and structural information, the intrinsic complexity of KG entities has been more or less overlooked. More concretely, we hypothesize KG entities may be more complex than we think, i.e., an entity may wear many hats and relational triplets may form due to more than a single reason. To this end, this paper proposes to learn disentangled representations of KG entities -a new method that disentangles the inner latent properties of KG entities. Our disentangled process operates at the graph level and a neighborhood mechanism is leveraged to disentangle the hidden properties of each entity. This disentangled representation learning approach is model agnostic and compatible with canonical KG embedding approaches. We conduct extensive experiments on several benchmark datasets, equipping a variety of models (DistMult, SimplE, and QuatE) with our proposed disentangling mechanism. Experimental results demonstrate that our proposed approach substantially improves performance on key metrics.

show abstract

“…For instance, several authors have focused on separating style (or sentiment) from content (John et al, 2019). In general, most existing approaches for text use some kind of supervision signal, such as aspect-specific similarity judgements (Jain et al, 2018) or sentiment labels (He et al, 2017).…”

Section: Related Workmentioning

confidence: 99%

A Mixture-of-Experts Model for Learning Multi-Facet Entity Embeddings

Alshaikh¹,

Bouraoui²,

Jeawak³

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

View full text Add to dashboard Cite

Various methods have already been proposed for learning entity embeddings from text descriptions. Such embeddings are commonly used for inferring properties of entities, for recommendation and entity-oriented search, and for injecting background knowledge into neural architectures, among others. Entity embeddings essentially serve as a compact encoding of a similarity relation, but similarity is an inherently multi-faceted notion. By representing entities as single vectors, existing methods leave it to downstream applications to identify these different facets, and to select the most relevant ones. In this paper, we propose a model that instead learns several vectors for each entity, each of which intuitively captures a different aspect of the considered domain. We use a mixture-of-experts formulation to jointly learn these facet-specific embeddings. The individual entity embeddings are learned using a variant of the GloVe model, which has the advantage that we can easily identify which properties are modelled well in which of the learned embeddings. This is exploited by an associated gating network, which uses pre-trained word vectors to encourage the properties that are modelled by a given embedding to be semantically coherent, i.e. to encourage each of the individual embeddings to capture a meaningful facet.

show abstract

Learning Disentangled Representations of Texts with Application to Biomedical Abstracts

Cited by 19 publications

References 27 publications

Learning Conceptual Spaces with Disentangled Facets

Learning Conceptual Spaces with Disentangled Facets

Knowledge Router: Learning Disentangled Representations for Knowledge Graphs

A Mixture-of-Experts Model for Learning Multi-Facet Entity Embeddings

Contact Info

Product

Resources

About