Higher-order Network Representation Learning

Rossi, Ryan A.; Ahmed, Nesreen K.; Koh, Eunyee

doi:10.1145/3184558.3186900

Cited by 80 publications

(45 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SNS [17] defines the structural similarity of two vertices based on graphlet and orbit, which can be combined with original methods of learning word2vec, such as CBOW. HONE [9] defines the problem of higher-order network representation learning based on network motifs. This approach defines a new adjacency matrix for each motif from two-node to fournode and demonstrates its effectiveness on link prediction.…”

Section: B Network Embeddingmentioning

confidence: 99%

MODEL: Motif-Based Deep Feature Learning for Link Prediction

Wang

Ren

et al. 2020

IEEE Trans. Comput. Soc. Syst.

View full text Add to dashboard Cite

Link prediction plays an important role in network analysis and applications. Recently, approaches for link prediction have evolved from traditional similarity-based algorithms into embedding-based algorithms. However, most existing approaches fail to exploit the fact that real-world networks are different from random networks. In particular, real-world networks are known to contain motifs, natural network building blocks reflecting the underlying network-generating processes. In this paper, we propose a novel embedding algorithm that incorporates network motifs to capture higher-order structures in the network. To evaluate its effectiveness for link prediction, experiments were conducted on three types of networks: social networks, biological networks, and academic networks. The results demonstrate that our algorithm outperforms both the traditional similarity-based algorithms (by 20%) and the stateof-the-art embedding-based algorithms (by 19%).

show abstract

Section: B Network Embeddingmentioning

confidence: 99%

MODEL: Motif-Based Deep Feature Learning for Link Prediction

Wang

Ren

et al. 2020

IEEE Trans. Comput. Soc. Syst.

View full text Add to dashboard Cite

show abstract

“…Higher-order structure in graphs has proven beneficial in many cases such as hierarchical object representations, scene understanding, link prediction and recommender systems [1,16,21]. Network Motifs [15] were first proposed to learn such higher-order embeddings through Random-Walk based models, and were extended to Graph Neural Network structures by designing a convolutional layer with Motif attention that could aggregate first-order neighborhood information as well as high-order Motif information [8]. Based on GCNs, the high-order normalized Laplacian matrix is leveraged to aggregate information passed from any-order neighboring nodes [1,9], though they fail to give a reasonable explanation as to why the high-order normalized Laplacian matrix should work in capturing connections across remote nodes.…”

Section: High-order Proximity Learning On Graphsmentioning

confidence: 99%

“…Graph Convolutional Networks(GCNs) [7] and its variants [4,[19][20][21] extend deep learning algorithms to graph-structured data by defining convolution operators on graphs, and have proven powerful when dealing with various downstream tasks [3,13,17,22], including learning low-dimensional embeddings of users and items in a recommender system [19,21,26]. However, such models struggle to capture higher-order connectivity patterns among nodes, as they only aggregate information from direct neighboring nodes (or firstorder neighbors), though it could be beneficial to take high-order connectivity into account [8,15]. Fig.…”

Section: Introductionmentioning

confidence: 99%

Stacked Mixed-Order Graph Convolutional Networks for Collaborative Filtering

Zhang¹,

McAuley²

2020

Proceedings of the 2020 SIAM International Conference on Data Mining

View full text Add to dashboard Cite

Graph-based recommendation algorithms treat useritem interactions as bipartite graphs, based on which low-dimensional vector representations of users and items seek to preserve the relationships among them. Previous methods usually capture users' preferences by directly learning first-order neighborhood patterns for each node, which limits their ability to exploit the similarity between two distant users/items as well as a user's preferences toward distant items. To address this potential weakness, in this paper, we propose SMOG-CF (Stacked Mixed-Order Graph Convolutional Networks for Collaborative Filtering), a GCN-based framework that can directly capture high-order connectivity among nodes. Instead of implicitly capturing high-order connectivity through embedding propagation, SMOG-CF facilitates 'path-level' information propagation between neighboring nodes at any order. The matrix form of our embedding propagation formulas yields a model that is easy to deploy and can be extended to a general framework by adopting various information construction and aggregation equations. Experiments on several datasets of varying scale demonstrate the efficacy of our model.

show abstract

“…But the method cannot be extended to handle heterogeneous networks. HONE [20] does not combine the best of both worlds − random walk based method that accounts for local neighborhood structure and motif-aware method that accounts for higher-order global network connectivity patterns, as we do. In addition, HONE (as well other existing methods) do not include the original network in the learning process, as we do.…”

Section: Introductionmentioning

confidence: 96%

“…However, no prior work has investigated the scope and impact of motifs in learning node embeddings for heterogeneous networks. Rossi et al introduced the problem of higher-order network representation learning using motifs for homogeneous networks [20]. But the method cannot be extended to handle heterogeneous networks.…”

Section: Introductionmentioning

confidence: 99%

motif2vec: Motif Aware Node Representation Learning for Heterogeneous Networks

Dareddy

Das

Yang

2019

2019 IEEE International Conference on Big Data (Big Data)

View full text Add to dashboard Cite

Recent years have witnessed a surge of interest in machine learning on graphs and networks with applications ranging from vehicular network design to IoT traffic management to social network recommendations. Supervised machine learning tasks in networks such as node classification and link prediction require us to perform feature engineering that is known and agreed to be the key to success in applied machine learning. Research efforts dedicated to representation learning, especially representation learning using deep learning, has shown us ways to automatically learn relevant features from vast amounts of potentially noisy, raw data. However, most of the methods are not adequate to handle heterogeneous information networks which pretty much represents most real world data today. The methods cannot preserve the structure and semantic of multiple types of nodes and links well enough, capture higherorder heterogeneous connectivity patterns, and ensure coverage of nodes for which representations are generated. In this paper, we propose a novel efficient algorithm, motif2vec that learns node representations or embeddings for heterogeneous networks. Specifically, we leverage higher-order, recurring, and statistically significant network connectivity patterns in the form of motifs to transform the original graph to motif graph(s), conduct biased random walk to efficiently explore higher order neighborhoods, and then employ heterogeneous skip-gram model to generate the embeddings. Unlike previous efforts that uses different graph meta-structures to guide the random walk, we use graph motifs to transform the original network and preserve the heterogeneity. We evaluate the proposed algorithm on multiple real-world networks from diverse domains and against existing state-of-theart methods on multi-class node classification and link prediction tasks, and demonstrate its consistent superiority over prior work.Author Terms − heterogeneous information networks, network embedding, network representation learning, feature learning, motifs

show abstract

Higher-order Network Representation Learning

Cited by 80 publications

References 10 publications

MODEL: Motif-Based Deep Feature Learning for Link Prediction

MODEL: Motif-Based Deep Feature Learning for Link Prediction

Stacked Mixed-Order Graph Convolutional Networks for Collaborative Filtering

motif2vec: Motif Aware Node Representation Learning for Heterogeneous Networks

Contact Info

Product

Resources

About