Modeling Dynamic Heterogeneous Network for Link Prediction using Hierarchical Attention with Temporal RNN

Abstract

Network embedding aims to learn low-dimensional representations of nodes while capturing structure information of networks. It has achieved great success on many tasks of network analysis such as link prediction and node classification. Most of existing network embedding algorithms focus on how to learn static homogeneous networks effectively. However, networks in the real world are more complex, e.g., networks may consist of several types of nodes and edges (called heterogeneous information) and may vary over time in terms of dynamic nodes and edges (called evolutionary patterns). Limited work has been done for network embedding of dynamic heterogeneous networks as it is challenging to learn both evolutionary and heterogeneous information simultaneously. In this paper, we propose a novel dynamic heterogeneous network embedding method, termed as DyHATR, which uses hierarchical attention to learn heterogeneous information and incorporates recurrent neural networks with temporal attention to capture evolutionary patterns. We benchmark our method on four real-world datasets for the task of link prediction. Experimental results show that DyHATR significantly outperforms several state-of-the-art baselines.

Dynamic Network Embedding Survey

“…In addition, heterogeneous dynamic graphs have also received more and more attention. [35] proposes DyHATR, which uses the hierarchical attention model to capture the heterogeneity and introduces the temporal attentive GRU/LSTM to model the evolutionary patterns among snapshots. While [36] focus on next-item recommendation problem, they learn embedings on temporal heterogenous User-Item bipartite network.…”

Heterogeneous Temporal Graph Neural Network

“…We replace the heterogeneous attention module with HGT [10] to avoid incorporating metapaths. DyHATR [15] uses hierarchical attention to learn heterogeneous information and incorporates RNNs with temporal attention to capture temporal dependencies.…”

“…In contrast, the graph structures remains unchanged for epidemiological networks, but the node features inevitably change with increased/decreased patient numbers. It is worth noting that dynamic heterogeneous graphs [14,15,16,17] can be treated as an instance of HTGs, where the dynamic nature comes from the evolving graph structures.…”

“…There exist some preliminary works. They could be roughly summarized into two categories: one first explores neural sequence models to process time-series features attached on each node, and then performs graph representation learning with the processed node features on the spatial domain [18,5,6,19]; the other first applies GNNs on each graph slice of a HTG, and then employs sequence models on the outputs of each slice to obtain the final representations [20,16,15]. Although these works could achieve satisfactory results, they are still faced with the following limitations: (1) The existing models are graph-dependent.…”

“…Spatial-temporal graphs [11,18,6,5] and dynamic graphs with homogeneous structures [12,30,13] have been widely studied in the literature. To further consider the graph heterogeneity, learning on dynamic heterogeneous graphs has drawn increasing attention, including dynamic heterogeneous graph embedding models [31,32,17,14] that solely consider graph structures and dynamic heterogeneous GNNs [15,20,33,34] that take both graph structure and node features into consideration. DyHATR [15] first introduces node-level and edge-level attentions to learn heterogeneous information and then applies RNNs with temporal attention to capture temporal dependencies.…”