Combining Two and Three-Way Embedding Models for Link Prediction in Knowledge Bases

Abstract

International audienceThis paper tackles the problem of endogenous link prediction for knowledge base completion. Knowledge bases can be represented as directed graphs whose nodes correspond to entities and edges to relationships. Previous attempts either consist of powerful systems with high capacity to model complex connectivity patterns, which unfortunately usually end up overfitting on rare relationships, or in approaches that trade capacity for simplicity in order to fairly model all relationships, frequent or not. In this paper, we propose Tatec, a happy medium obtained by complementing a high-capacity model with a simpler one, both pre-trained separately and then combined. We present several variants of this model with different kinds of regularization and combination strategies and show that this approach outperforms existing methods on different types of relationships by achieving state-of-the-art results on four benchmarks of the literature

An Interpretable Knowledge Transfer Model for Knowledge Base Completion

“…The overall distribution of relation frequencies resembles that of word frequencies, subject to the zipf's law. Since the frequencies of relations approximately follow a power distribution, their log Model Additional Information WN18 FB15k Mean Rank Hits@10 Mean Rank Hits@10 SE (Bordes et al, 2011) No 985 80.5 162 39.8 Unstructured (Bordes et al, 2014) No 304 38.2 979 6.3 TransE (Bordes et al, 2013) No 251 89.2 125 47.1 TransH (Wang et al, 2014) No 303 86.7 87 64.4 TransR (Lin et al, 2015b) No 225 92.0 77 68.7 CTransR (Lin et al, 2015b) No 218 92.3 75 70.2 KG2E No 348 93.2 59 74.0 TransD No 212 92.2 91 77.3 TATEC (García-Durán et al, 2016) No --58 76.7 NTN (Socher et al, 2013) No -66.1 -41.4 DISTMULT (Yang et al, 2015) No Shen et al (2016), we divide the models into two groups. The first group contains intrinsic models without using extra information.…”

Neighborhood Mixture Model for Knowledge Base Completion

“…than previously reported when the TransE used for initalization performs as well as reported in this paper. Furthermore, García-Durán et al (2015), Lin et al (2015a), García-Durán et al (2016) and Nickel et al (2016b) also showed that for entity prediction TransE obtains very competitive results which are much higher than the TransE results originally published in Bordes et al (2013). 3 As presented in Table 3, for entity prediction using WN11, TransE-NMM with the filtering threshold τ = 10 only obtains better mean rank than TransE (about 15% relative improvement) but lower Hits@10 and mean reciprocal rank.…”

“…TransE (Bordes et al, 2013) is a simple embedding model for knowledge base completion, which, despite of its simplicity, obtains very competitive results (García-Durán et al, 2016;Nickel et al, 2016b). In TransE, both entities e and relations r are represented with k-dimensional vectors v e ∈ R k and v r ∈ R k , respectively.…”

Multilingual Relation Extraction using Compositional Universal Schema

“…In order to generalize well, prediction is often posed as low-rank matrix or tensor factorization. A variety of model variants have been suggested, where the probability of a given edge existing depends on a multi-linear form (Nickel et al, 2011;García-Durán et al, 2015;Bordes et al, 2013;, or non-linear interactions between s, r, and o (Socher et al, 2013). Other approaches model the compositionality of multi-hop paths, typically for question answering Gu et al, 2015;Neelakantan et al, 2015).…”