RotateQVS: Representing Temporal Information as Rotations in Quaternion Vector Space for Temporal Knowledge Graph Completion

Abstract

Temporal factors are tied to the growth of facts in realistic applications, such as the progress of diseases and the development of political situation, therefore, research on Temporal Knowledge Graph (TKG) attracks much attention. In TKG, relation patterns inherent with temporality are required to be studied for representation learning and reasoning across temporal facts. However, existing methods can hardly model temporal relation patterns, nor can capture the intrinsic connections between relations when evolving over time, lacking of interpretability. In this paper, we propose a novel temporal modeling method which represents temporal entities as Rotations in Quaternion Vector Space (RotateQVS) and relations as complex vectors in Hamilton's quaternion space. We demonstrate our method can model key patterns of relations in TKG, such as symmetry, asymmetry, inverse, and can further capture time-evolved relations by theory. Empirically, we show that our method can boost the performance of link prediction tasks over four temporal knowledge graph benchmarks.

HGE: Embedding Temporal Knowledge Graphs in a Product Space of Heterogeneous Geometric Subspaces

“…Embeddings for temporal knowledge graphs must account for temporal facts including time components and express corresponding temporal patterns. Four kinds of logical patterns, symmetric, inverse, asymmetric and evolve are mostly considered and studied in existing TKGE models (Chen et al 2022;Xu et al 2020). However, their definitions either neglect time information or merely consider patterns when facts happen at the same time.…”

“…Existing embedding approaches such as TeRO, Rotate-QVS, and TLT-KGE (Xu et al 2020;Chen et al 2022;Zhang et al 2022) resorted to single underlying embedding spaces, such as Complex space or Quaternion space to model symmetric patterns by the rotations on a unit hypersphere. Other works (Chami et al 2020;Balazevic, Allen, and Hospedales 2019;Montella, Barahona, and Heinecke 2021;Han et al 2020) use hyperbolic space to preserve hierarchical patterns in temporal KGs.…”

Extrapolation over temporal knowledge graph via hyperbolic embedding

“…where 1 ≤ i ≤ l. For ease of computation, we map Q r ð⋅Þ ∈ B n;c r to the tangent space of the origin of Poincaré ball model shared by all c r for r ∈ R according to the exponential map and logarithmic map in Equation (3). Given that cooccurrence events are consistent and structurally dependant, we leverage mean-pooling to aggregate information from events involving o.…”

“…To represent knowledge that is time-dependent and evolving, previous methods [1][2][3] attempted to encode quadruplets with time-specific embeddings of entities, relations and timestamps. Extending the scope of knowledge representation, these methods capture rich semantic patterns including symmetry, asymmetry, inversion and time evolution to some extent.…”

LambdaKG: A Library for Pre-trained Language Model-Based Knowledge Graph Embeddings

“…By comparison with embedding-based KGE approaches, text-based methods incorporate available texts for KGE. With the development of Pre-trained Language Models (PLMs), many text-based models Saxena et al, 2022;Kim et al, 2020;Markowitz et al, 2022;Chen et al, 2022a; have been proposed, which can obtain promising performance and take advantage of allocating a fixed memory footprint for largescale real-world KGs. Recently, large language models (LLMs) (e.g., GPT-3 (Brown et al, 2020), ChatGPT (OpenAI, 2022)) further demonstrated the ability to perform a variety of natural language processing (NLP) tasks without adaptation, providing potential opportunities of better knowledge representations.…”