Representing Text for Joint Embedding of Text and Knowledge Bases

Abstract

Models that learn to represent textual and knowledge base relations in the same continuous latent space are able to perform joint inferences among the two kinds of relations and obtain high accuracy on knowledge base completion (Riedel et al., 2013). In this paper we propose a model that captures the compositional structure of textual relations, and jointly optimizes entity, knowledge base, and textual relation representations. The proposed model significantly improves performance over a model that does not share parameters among textual relations with common sub-structure.

Distributed Prediction of Relations for Entities: The Easy, The Difficult, and The Impossible

“…While the lack of contextual support seems more fundamental, it could be addressed by either using syntax-based embeddings (Levy and Goldberg, 2014a) that can better pick up the specific context patterns characteristic for these relations, or by optimizing the input word embeddings for the task. This becomes a similar problem to joint training of representations from knowledge base structure and textual evidence (Perozzi et al, 2014;Toutanova et al, 2015).…”

An Insight Extraction System on BioMedical Literature with Deep Neural Networks

“…There are recent work with the use of deep reinforcement learning on healthcare study (Li, 2017). Our approach is inspired by recent embedding learning work to jointly represent texts and knowledge base (Toutanova et al, 2015(Toutanova et al, , 2016, previous work on embedding transfer learning (Bordes et al, 2013) and noisecontrastive estimation . Lastly our work models insight extraction as a similarity measurement problem, and is inspired by similarity measurement work on pairwise word interaction modeling with deep neural networks.…”

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

“…propose the Neural Tensor Network (NTN), and Yang et al (2015) the Bilinear model using this technique. Other approaches modify the objective function or change the structure of the model in order to integrate distributional and relational information Fried and Duh, 2015;Toutanova and Chen, 2015). retrofit word vectors after they are trained according to distributional criteria.…”

“…Since these methods did not use any co-occurrence information from a text corpus, all entities were required to appear at least once in the training data, ruling out generalization to unseen entities 1 . More recently, Xu et al (2014) combined the training objective of SKIP-GRAM with the training objective of (Bordes et al, 2013) to incorporate lexical 1 There exists work on relation extraction and knowledgebase completion that combines structured relation triplets and logical rules with unstructured text using various forms of latent variable models (Riedel et al, 2013;Chang et al, 2014;Toutanova et al, 2015;Rocktäschel et al, 2015). 506 knowledge into word embeddings.…”