Graph Convolutional Networks with Motif-based Attention

Lee, John Boaz; Rossi, Ryan A.; Kong, Xiangnan; Kim, Sung‐Chul; Koh, Eunyee; Rao, Anup

doi:10.1145/3357384.3357880

Cited by 84 publications

(50 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The concept of motif was first introduced in 2002 [10] , which represents the frequently repeated patterns in complex networks and is the building block of complex networks. Some work [11][12][13] proves that motif plays an important role in understanding and capturing higher-order structure information of the biological networks, social networks, academic networks, and so on. Capturing the motif structure and its interaction can improve the quality of network embedding.…”

Section: Deep Graph Convolutional Neural Networkmentioning

confidence: 99%

LGL-GNN: Learning Global and Local Information for Graph Neural Networks

Wang

Cui

et al. 2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

In this article, we have developed a graph convolutional network model LGL that can learn global and local information at the same time for effective graph classification tasks. Our idea is to concatenate the convolution results of the deep graph convolutional network and the motif-based subgraph convolutional network layer by layer, and give attention weights to global features and local features. We hope that this method can alleviate the over-smoothing problem when the depth of the neural networks increases, and the introduction of motif for local convolution can better learn local neighborhood features with strong connectivity. Finally, our experiments on standard graph classification benchmarks prove the effectiveness of the model.

show abstract

Section: Deep Graph Convolutional Neural Networkmentioning

confidence: 99%

LGL-GNN: Learning Global and Local Information for Graph Neural Networks

Wang

Cui

et al. 2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Standard MPNNs have been shown to be at most as powerful as the WL test in distinguishing non-isomorphic graphs [10], [11]. Some works aggregate other types of structures instead of neighbors such as small subgraphs [25] or paths [26]. Considerable efforts have also been devoted to building deeper MPNNs [27], [28] and more powerful models [11], [12].…”

Section: Related Workmentioning

confidence: 99%

Permute Me Softly: Learning Soft Permutations for Graph Representations

Nikolentzos¹,

Dasoulas²,

Vazirgiannis³

2021

Preprint

View full text Add to dashboard Cite

Graph neural networks (GNNs) have recently emerged as a dominant paradigm for machine learning with graphs.Research on GNNs has mainly focused on the family of message passing neural networks (MPNNs). Similar to the Weisfeiler-Leman (WL) test of isomorphism, these models follow an iterative neighborhood aggregation procedure to update vertex representations, and they next compute graph representations by aggregating the representations of the vertices. Although very successful, MPNNs have been studied intensively in the past few years. Thus, there is a need for novel architectures which will allow research in the field to break away from MPNNs. In this paper, we propose a new graph neural network model, so-called π-GNN which learns a "soft" permutation (i. e., doubly stochastic) matrix for each graph, and thus projects all graphs into a common vector space. The learned matrices impose a "soft" ordering on the vertices of the input graphs, and based on this ordering, the adjacency matrices are mapped into vectors. These vectors can be fed into fully-connected or convolutional layers to deal with supervised learning tasks. In case of large graphs, to make the model more efficient in terms of running time and memory, we further relax the doubly stochastic matrices to row stochastic matrices. We empirically evaluate the model on graph classification and graph regression datasets and show that it achieves performance competitive with state-of-the-art models.

show abstract

“…Graph Attention Networks (GATs) have been successfully applied to various tasks over graphs (Velickovic et al, 2018;Lee et al, 2018b), such as graph classification (Wu et al, 2019b;Lee et al, 2018a), link prediction (Abu-El-Haija et al, 2018), and node classification (Lee et al, 2019;Zhang et al, 2020a). GATs learn from the underlying graph structure by making use of localized attention mechanism (Wu et al, 2019a;Xu et al, 2019;Vashishth et al, 2020b), where the hidden representation of each node is computed by recursively aggregating and attending over its corresponding local neighbors' features, and the weighting coefficients are calculated inductively with self-attention strategy (Thekumparampil et al, 2018;Qian et al, 2018;Zhang et al, 2018).…”

Section: Introductionmentioning

confidence: 99%

Incorporating Global Information in Local Attention for Knowledge Representation Learning

Yu¹,

Zhou²,

Xie³

et al. 2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

View full text Add to dashboard Cite

Graph Attention Networks (GATs) have proven a promising model that takes advantage of localized attention mechanism to perform knowledge representation learning (KRL) on graph-structure data, e.g., Knowledge Graphs (KGs). While such approaches model entities' local pairwise importance, they lack the capability to model global importance relative to other entities of KGs. This causes such models to miss critical information in tasks where global information is also a significant component for the task, such as in knowledge representation learning. To address the issue, we allow the proper incorporation of global information into the GAT family of models through the use of scaled entity importance, which is calculated by an attention-based global random walk algorithm. In the context of KRL, incorporating global information boosts performance significantly. Experimental results on KG entity prediction against the state-of-thearts sufficiently demonstrate the effectiveness of our proposed model.

show abstract

Graph Convolutional Networks with Motif-based Attention

Cited by 84 publications

References 17 publications

LGL-GNN: Learning Global and Local Information for Graph Neural Networks

LGL-GNN: Learning Global and Local Information for Graph Neural Networks

Permute Me Softly: Learning Soft Permutations for Graph Representations

Incorporating Global Information in Local Attention for Knowledge Representation Learning

Contact Info

Product

Resources

About