Using Pairwise Occurrence Information to Improve Knowledge Graph Completion on Large-Scale Datasets

Abstract

Bilinear models such as DistMult and ComplEx are effective methods for knowledge graph (KG) completion. However, they require large batch sizes, which becomes a performance bottleneck when training on large scale datasets due to memory constraints. In this paper we use occurrences of entity-relation pairs in the dataset to construct a joint learning model and to increase the quality of sampled negatives during training. We show on three standard datasets that when these two techniques are combined, they give a significant improvement in performance, especially when the batch size and the number of generated negative examples are low relative to the size of the dataset. We then apply our techniques to a dataset containing 2 million entities and demonstrate that our model outperforms the baseline by 2.8% absolute on hits@1.1 The choice of loss function has a very strong effect on the optimal negative ratio, but with any loss function, larger batches tend to improve the results.

Incorporating Global Information in Local Attention for Knowledge Representation Learning

“…In recent years, knowledge representation learning on KGs has been a hot research topic (Xiao et al, 2017;Shi and Weninger, 2017;Ebisu and Ichise, 2019;Balazevic et al, 2019;Zhang et al, 2020b). These methods roughly fall into four categories: (i) Translational-based models, which view relations as translations from a head entity to a tail entity, such as Trans(E, H, R, D and G) (Bordes et al, 2013;Lin et al, 2015;Ji et al, 2015;Xiao et al, 2016), ComplEx (Trouillon et al, 2016), JoBi ComplEx (Balkir et al, 2019).…”