Learning from History: Modeling Temporal Knowledge Graphs with Sequential Copy-Generation Networks

Abstract

Large knowledge graphs often grow to store temporal facts that model the dynamic relations or interactions of entities along the timeline. Since such temporal knowledge graphs often suffer from incompleteness, it is important to develop time-aware representation learning models that help to infer the missing temporal facts. While the temporal facts are typically evolving, it is observed that many facts often show a repeated pattern along the timeline, such as economic crises and diplomatic activities. This observation indicates that a model could potentially learn much from the known facts appeared in history. To this end, we propose a new representation learning model for temporal knowledge graphs, namely CyGNet, based on a novel time-aware copy-generation mechanism. CyGNet is not only able to predict future facts from the whole entity vocabulary, but also capable of identifying facts with repetition and accordingly predicting such future facts with reference to the known facts in the past. We evaluate the proposed method on the knowledge graph completion task using five benchmark datasets. Extensive experiments demonstrate the effectiveness of CyGNet for predicting future facts with repetition as well as de novo fact prediction.

IMF: Interpretable Multi-Hop Forecasting on Temporal Knowledge Graphs

“…We compare the performance of IMR-TransE against the temporal KG reasoning models, including TTransE [ 34 ], TA-DistMult/TA-TransE [ 30 ], DE-SimplE [ 39 ], TNTComplEx [ 32 ], CyGNet [ 11 ], RE-Net [ 38 ], TANGO [ 40 ], TITer [ 14 ], and xERTR [ 7 ].…”

“…Temporal KGs import the time dimension into static KGs, which makes the facts of a specific timestamp extremely sparse. The temporal KG reasoning task can be divided into two categories: reasoning about historical facts [ 4 , 5 , 6 , 7 , 8 , 30 ], i.e., interpolation on temporal KGs, and reasoning about future facts [ 3 , 4 , 7 , 11 ], i.e., forecasting on temporal KGs. The former predicts the missing facts of a specific historical moment based on the facts of all moments, and the latter predicts future events based only on the past facts.…”

“…Most current research on temporal KG completion focuses on interpolation [ 4 , 5 , 6 , 7 , 8 , 9 , 10 ]. Recently, there have been attempts to investigate temporal KG forecasting [ 3 , 4 , 7 , 11 , 12 , 13 ]. According to the interpretability, research on temporal KG forecasting can be divided into two categories.…”

“…The other type is interpretable approaches. CyGNet [ 11 ] utilizes one-hop repetitive facts to realize prediction. Its performance is limited by the lack of direct repetitive knowledge of historical moments.…”

Modeling the evolution of temporal knowledge graphs with uncertainty

“…We compare our model against nine baselines, including three static ones: Distmult (Yang et al [2014]), TuckER (Balazevic et al [2019]), and COMPGCN (Vashishth et al [2019]). Furthermore, we report 1 on six tKG baselines: TTransE (Leblay and Chekol [2018]), TA-DistMult (García-Durán et al [2018]), CyGNet (Zhu et al [2020]), DE-SimplE (Goel et al [2020]), TNTComplEx (Lacroix et al [2020]), RE-NET (Jin et al [2020]). For both data sets, we provide time-aware filtered results, following the evaluation setting of Han et al [2020].…”