Meta-Graph Based HIN Spectral Embedding: Methods, Analyses, and Insights

Abstract

Heterogeneous information network (HIN) has drawn significant research attention recently, due to its power of modeling multi-typed multi-relational data and facilitating various downstream applications. In this decade, many algorithms have been developed for HIN modeling, including traditional similarity measures and recent embedding techniques. Most algorithms on HIN leverage meta-graphs or meta-paths (special cases of meta-graphs) to capture various semantics. Given any arbitrary set of meta-graphs, existing algorithms either consider them as equally important or study their different importance through supervised learning. Their performance largely relies on prior knowledge and labeled data. While unsupervised embedding has shown to be a fundamental solution for various homogeneous network mining tasks, for HIN, it is a much harder problem due to such a presence of various meta-graphs.In this work, we propose to study the utility of different meta-graphs, as well as how to simultaneously leverage multiple meta-graphs for HIN embedding in an unsupervised manner. Motivated by prolific research on homogeneous networks, especially spectral graph theory, we firstly conduct a systematic empirical study on the spectrum and embedding quality of different meta-graphs on multiple HINs, which leads to an efficient method of meta-graph assessment. It also helps us to gain valuable insight into the higher-order organization of HINs and indicates a practical way of selecting useful embedding dimensions. Further, we explore the challenges of combining multiple meta-graphs to capture the multi-dimensional semantics in HIN through reasoning from mathematical geometry and arrive at an embedding compression method of autoencoder with 2,1-loss, which finds the most informative meta-graphs and embeddings in an end-to-end unsupervised manner. Finally, empirical analysis suggests a unified workflow to close the gap between our meta-graph assessment and combination methods. To the best of our knowledge, this is the first research effort to provide rich theoretical and empirical analyses on the utility of meta-graphs and their combinations, especially regarding HIN embedding. Extensive experimental comparisons with various state-of-the-art neural network based embedding methods on multiple real-world HINs demonstrate the effectiveness and efficiency of our framework in finding useful meta-graphs and generating high-quality HIN embeddings.

Lime: Low-Cost and Incremental Learning for Dynamic Heterogeneous Information Networks

“…Deep network embedding models. Previous network representation learning models primarily focus on improving the learning ability such as preserving original network structural information and properties [3] or semantic correlations of different types of nodes and relationships [31], [12], [41]. Unlike these, LIME aims to improve the efficiency in computational resources and training time to allow the learning algorithm to scale to large networks and to respond to a dynamically changing HIN quickly.…”

Churn modeling with probabilistic meta paths-based representation learning

“…Several other relevant works on heterogeneous networks have been proposed (Sun et al, 2011;Chang et al, 2015;Tang et al, 2015a;Chen & Sun, 2017;Fu et al, 2017;Jiang et al, 2017;Shi et al, 2018;Yang et al, 2018). The work by Sun et al (2011) is also based on meta paths but it focuses on defining a similarity measure instead of learning representations.…”

“…Shi et al (2018) proposes learning different representations per node to address different aspects of nodes encapsulated within the heterogeneous network which is an interesting idea but is out of the scope of this paper. The idea of combining and reducing heterogeneous information is also presented in Yang et al (2018) but they focus on meta graphs rather than meta paths and use deep denoising autoencoders for unsupervised dimensionality reduction.…”

Neural Embedding Propagation on Heterogeneous Networks

“…To capture such complex interactions, the tool of meta-path has been proposed and leveraged by most existing models on heterogeneous networks [18]. Traditional object proximity models measure the total strength of various interactions by counting the number of instances of different meta-paths between objects and adding up the counts with pre-defined or learned weights [18], [23], [19], [37], [22], [21], [38], [14], [39], whereas the more recent network representation learning methods leverage meta-path guided random walks to jointly model multiple interactions in a latent embedding space [20], [40], [15], [41], [13], [42]. However, the consideration of a fixed set of metapaths, while helping regulate the complex interactions, largely relies on the quality of the meta-paths under consideration, and limits the flexibility of the model, which is unable to handle any interactions not directly captured by the meta-paths.…”