Network Representation Learning: A Survey

Abstract

With the widespread use of information technologies, information networks are becoming increasingly popular to capture complex relationships across various disciplines, such as social networks, citation networks, telecommunication networks, and biological networks. Analyzing these networks sheds light on different aspects of social life such as the structure of societies, information diffusion, and communication patterns. In reality, however, the large scale of information networks often makes network analytic tasks computationally expensive or intractable. Network representation learning has been recently proposed as a new learning paradigm to embed network vertices into a low-dimensional vector space, by preserving network topology structure, vertex content, and other side information. This facilitates the original network to be easily handled in the new vector space for further analysis. In this survey, we perform a comprehensive review of the current literature on network representation learning in the data mining and machine learning field. We propose new taxonomies to categorize and summarize the state-of-the-art network representation learning techniques according to the underlying learning mechanisms, the network information intended to preserve, as well as the algorithmic designs and methodologies. We summarize evaluation protocols used for validating network representation learning including published benchmark datasets, evaluation methods, and open source algorithms. We also perform empirical studies to compare the performance of representative algorithms on common datasets, and analyze their computational complexity. Finally, we suggest promising research directions to facilitate future study.Definition 2 (First-order Proximity). The first-order proximity is the local pairwise proximity between two connected vertices [1]. For each vertex pair (v i , v j ), if (v i , v j ) ∈ E, the first-order proximity between v i and v j is w ij ; otherwise, the first-order proximity between v i and v j is 0. The first-order proximity captures the direct neighbor relationships between vertices.Definition 3 (Second-order Proximity and High-order Proximity). The second-order proximity captures the 2-step relations between each pair of vertices [1]. For each vertex pair (v i , v j ), the second order proximity is determined by the number of common neighbors shared by the two vertices, which can also be measured by the 2-step transition probability from v i to v j equivalently. Compared with the second-order proximity, the highorder proximity [26] captures more global structure, which explores k-step (k ≥ 3) relations between each pair of vertices. For each vertex pair (v i , v j ), the higherorder proximity is measured by the k-step (k ≥ 3) transition probability from vertex v i to vertex v j , which can also be reflected by the number of k-step (k ≥ 3) paths from v i to v j . The second-order and high-order proximity capture the similarity between a pair of, indirectly connected, vertices with similar structural contex...

Gravity-Inspired Graph Autoencoders for Directed Link Prediction

“…Graphs are useful data structures to efficiently represent relationships among items. Due to the proliferation of graph data [54,56], a large variety of specific problems initiated significant research efforts from the Machine Learning community, aiming at extracting relevant information from such structures. This includes node clustering [33], influence maximization [21], graph generation [45] and link prediction, on which we focus in this paper.…”

“…The Adamic-Adar and Katz indices [29], reflecting neighborhood structure and node proximity, are notorious examples of such similarity indices. More recently, along with the increasing efforts in extending Deep Learning methods to graph structures [6,43,54], these approaches have been outperformed by the node embedding paradigm [16,46,56]. In a nutshell, the strategy is to train graph neural networks to represent nodes as vectors in a low-dimensional vector space, namely the embedding space.…”

Evolutionary Computation, Optimization, and Learning Algorithms for Data Science

“…If PCA does not yield a better result, it means that features are not correlated or have non-linear relationships. However, researchers often used to enable data easy to explore and visualize, in case for representation learning (RL) [28].…”

Kernel Node Embeddings

“…With the advancements in data production, storage and consumption, networks are becoming omnipresent; data from diverse disciplines can be represented as graph structures with prominent examples here being various social, information, technological and biological networks. Developing machine learning algorithms to analyze, predict and make sense of the structure of graph data has become a crucial task with a plethora of cross-disciplinary applications [1], [2]. The major challenge in machine learning on graph data concerns the encoding of information about its structural properties into the learning model.…”