Network Embedding as Matrix Factorization

Abstract

Since the invention of word2vec [28,29], the skip-gram model has significantly advanced the research of network embedding, such as the recent emergence of the DeepWalk, LINE, PTE, and node2vec approaches. In this work, we show that all of the aforementioned models with negative sampling can be unified into the matrix factorization framework with closed forms. Our analysis and proofs reveal that: (1) DeepWalk [31] empirically produces a low-rank transformation of a network's normalized Laplacian matrix;(2) LINE [37], in theory, is a special case of DeepWalk when the size of vertices' context is set to one; (3) As an extension of LINE, PTE [36] can be viewed as the joint factorization of multiple networks' Laplacians; (4) node2vec [16] is factorizing a matrix related to the stationary distribution and transition probability tensor of a 2nd-order random walk. We further provide the theoretical connections between skip-gram based network embedding algorithms and the theory of graph Laplacian. Finally, we present the NetMF method 1 as well as its approximation algorithm for computing network embedding. Our method offers significant improvements over DeepWalk and LINE for conventional network mining tasks. This work lays the theoretical foundation for skip-gram based network embedding methods, leading to a better understanding of latent network representation learning.

Improve Network Embeddings with Regularization

“…We use seven datasets in our experiment. Some data is widely used to benchmark the network embedding algorithms [2][3][4][6][7][8][9]. The statistics of the datasets are listed in Table 1.…”

Returning the N to NLP: Towards Contextually Personalized Classification Models

“…Improving NLP tasks with social graphs An emerging line of research makes use of social interactions to derive information about the userrepresenting each user as a node in a social graph and creating low dimensional user embeddings induced by neural architecture (Grover and Leskovec, 2016;Qiu et al, 2018). Including network information improves performance on profiling tasks such as predicting user gender (Farnadi et al, 2018) or occupation , as well as on detecting online behavior such as cyberbullying (Mathur et al, 2018), abusive language use (Qian et al, 2018;Mishra et al, 2018) or suicide ideation (Mishra et al, 2019).…”

The Entropy of Graph Embeddings: A Proxy of Potential Mobility in Covid19 Outbreaks

“…Following the standard factorization approach for network embedding [6], the latent representations for nodes of G = (V, E) are obtained from the SVD ofM G = log(max(M G , 1)), where M G is the Pointwise Mutual Information (PMI) matrix. More recently, Qiu et al [9] show that M G can be posed in the following terms…”