Convolutional 2D Knowledge Graph Embeddings

Abstract

Link prediction for knowledge graphs is the task of predicting missing relationships between entities. Previous work on link prediction has focused on shallow, fast models which can scale to large knowledge graphs. However, these models learn less expressive features than deep, multi-layer models — which potentially limits performance. In this work we introduce ConvE, a multi-layer convolutional network model for link prediction, and report state-of-the-art results for several established datasets. We also show that the model is highly parameter efficient, yielding the same performance as DistMult and R-GCN with 8x and 17x fewer parameters. Analysis of our model suggests that it is particularly effective at modelling nodes with high indegree — which are common in highly-connected, complex knowledge graphs such as Freebase and YAGO3. In addition, it has been noted that the WN18 and FB15k datasets suffer from test set leakage, due to inverse relations from the training set being present in the test set — however, the extent of this issue has so far not been quantified. We find this problem to be severe: a simple rule-based model can achieve state-of-the-art results on both WN18 and FB15k. To ensure that models are evaluated on datasets where simply exploiting inverse relations cannot yield competitive results, we investigate and validate several commonly used datasets — deriving robust variants where necessary. We then perform experiments on these robust datasets for our own and several previously proposed models, and find that ConvE achieves state-of-the-art Mean Reciprocal Rank across all datasets.

IFGAN: Information fusion generative adversarial network for knowledge base completion

“…The function of the generator is to provide high-quality negative triplets to improve the discriminator. The convolution structure is proven to be efficient in KBC (Dettmers et al, 2018). To our knowledge, we are the first to apply a convolution structure to generate negative triplets.…”

“…The sufficient feature interaction brought from convolutional layers can also generate high-quality negative triplets to deceive the discriminator. Here, we use ConvE (Dettmers et al, 2018) to exemplify the feasibility of the structure, while we attempted depthwise separable convolution (Chollet, 2017) and involution (Li et al, 2021) in the extended experiments. The score function in the generator is given as:…”

“…The convolutional generator effectively provides high-quality negative samples. In this figure, we use ConvE (Dettmers et al, 2018) to illustrate the convolutional generator. Different kinds of convolution methods can be used here.…”

“…Different from the generative models designed for computer vision that use upsampling in the convolution structure, IFGAN uses a convolution layer to ensure more feature interaction. In the default setting introduced in a previous article and illustrated in Figure 6, we use a simple convolution structure such as ConvE (Dettmers et al, 2018). Here, we perform an extended experiment with a model using depthwise separable convolution (Chollet, 2017) and involution (Li et al, 2021).…”

“…Previous works considered each triplet independently without taking into account the relationships between triplets; these GNN-based models (Hamaguchi et al, 2017;Veličkovi c et al, 2017) aggregate local information in the graph neighbourhood for each node. CNN-based models (Dettmers et al, 2018;Junsheng Xiao et al, 2021;Vashishth et al, 2020) are recognized for their high-parameter efficiency. They use convolution layers to learn embedding under the assumption that the performance was otherwise limited by feature interactions.…”