RelWalk - A Latent Variable Model Approach to Knowledge Graph Embedding

Abstract

Embedding entities and relations of a knowledge graph in a low-dimensional space has shown impressive performance in predicting missing links between entities. Although progresses have been achieved, existing methods are heuristically motivated and theoretical understanding of such embeddings is comparatively underdeveloped. This paper extends the random walk model (Arora et al., 2016a) of word embeddings to Knowledge Graph Embeddings (KGEs) to derive a scoring function that evaluates the strength of a relation R between two entities h (head) and t (tail). Moreover, we show that marginal loss minimisation, a popular objective used in much prior work in KGE, follows naturally from the loglikelihood ratio maximisation under the probabilities estimated from the KGEs according to our theoretical relationship. We propose a learning objective motivated by the theoretical analysis to learn KGEs from a given knowledge graph. Using the derived objective, accurate KGEs are learnt from FB15K237 and WN18RR benchmark datasets, providing empirical evidence in support of the theory. * Danushka Bollegala holds concurrent appointments as a Professor at University of Liverpool and as an Amazon Scholar. This paper describes work performed at the University of Liverpool and is not associated with Amazon.

Learning to Borrow -- Relation Representation for Without-Mention Entity-Pairs for Knowledge Graph Completion

“…NeighbBorrow: Given a without-mention entitypair (h * , t * ), we can borrow the LDPs from the first nearest neighbouring (1NN) with-mention entitypair (h, t). The similarity between entity-pairs can be computed using ( 4) in an unsupervised manner using pretrained entity embeddings such as Rel-Walk embeddings (Bollegala et al, 2021).…”