Data Augmentation for Graph Neural Networks

Zhao, Tong; Liu, Yozen; Neves, Lucio Pereira; Woodford, Oliver J.; Jiang, Meng; Shah, Neil

doi:10.1609/aaai.v35i12.17315

Cited by 139 publications

(39 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…All the baselines and our proposed method can be applied to all types of networks. We adopt the same data split of Cora and Citeseer as [ 21 ], and a split of training, validation, and testing with a ratio of 10:20:70 on other datasets [ 12 ].…”

Section: Methodsmentioning

confidence: 99%

“…NeuralSparse [ 11 ] considers the graph sparsification task by removing irrelevant edges. GAUG [ 12 ] utilizes a GNN to parameterize the categorical distribution instead of MLP in NerualSparse. PTDNet [ 13 ] prunes task-irrelevant edges by penalizing the number of edges in the sparsified graph with parameterized networks.…”

Section: Related Workmentioning

confidence: 99%

“…Figure 1 a); and (2) the attention cannot be easily trained well due to the limited labeled data [ 10 ]. Noise detection incorporates an edge classifier to estimate the probability of inter-class connection for each edge [ 11 , 12 , 13 , 14 , 15 , 16 ]. Although it can be better trained owing to the supervision of edge labels, the edge classifier also suffers from local structure heterogeneity and lacks consideration of global information.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Generic Structure Extraction with Bi-Level Optimization for Graph Structure Learning

Yin

Luo

2022

Entropy

View full text Add to dashboard Cite

Currently, most Graph Structure Learning (GSL) methods, as a means of learning graph structure, improve the robustness of GNN merely from a local view by considering the local information related to each edge and indiscriminately applying the mechanism across edges, which may suffer from the local structure heterogeneity of the graph (i.e., the uneven distribution of inter-class connections over nodes). To overcome the drawbacks, we extract the graph structure as a learnable parameter and jointly learn the structure and common parameters of GNN from the global view. Excitingly, the common parameters contain the global information for nodes features mapping, which is also crucial for structure optimization (i.e., optimizing the structure relies on global mapping information). Mathematically, we apply a generic structure extractor to abstract the graph structure and transform GNNs in the form of learning structure and common parameters. Then, we model the learning process as a novel bi-level optimization, i.e., Generic Structure Extraction with Bi-level Optimization for Graph Structure Learning (GSEBO), which optimizes GNN parameters in the upper level to obtain the global mapping information and graph structure is optimized in the lower level with the global information learned from the upper level. We instantiate the proposed GSEBO on classical GNNs and compare it with the state-of-the-art GSL methods. Extensive experiments validate the effectiveness of the proposed GSEBO on four real-world datasets.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Generic Structure Extraction with Bi-Level Optimization for Graph Structure Learning

Yin

Luo

2022

Entropy

View full text Add to dashboard Cite

show abstract

“…Therefore, only a few works consider graph data augmentation. [60] note that a node classification task can be perfectly solved if edges only exist between same class samples. They increase homophily by adding edges between nodes that a neural network predicts belong to the same class and breaking edges between nodes of predicted dissimilar classes.…”

Section: F Dataset Generation and Experimental Detailsmentioning

confidence: 99%

Analyzing Data-Centric Properties for Graph Contrastive Learning

Trivedi¹,

Lubana²,

Heimann³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent analyses of self-supervised learning (SSL) find the following data-centric properties to be critical for learning good representations: invariance to taskirrelevant semantics, separability of classes in some latent space, and recoverability of labels from augmented samples. However, given their discrete, non-Euclidean nature, graph datasets and graph SSL methods are unlikely to satisfy these properties. This raises the question: how do graph SSL methods, such as contrastive learning (CL), work well? To systematically probe this question, we perform a generalization analysis for CL when using generic graph augmentations (GGAs), with a focus on data-centric properties. Our analysis yields formal insights into the limitations of GGAs and the necessity of task-relevant augmentations. As we empirically show, GGAs do not induce task-relevant invariances on common benchmark datasets, leading to only marginal gains over naive, untrained baselines. Our theory motivates a synthetic data generation process that enables control over task-relevant information and boasts pre-defined optimal augmentations. This flexible benchmark helps us identify yet unrecognized limitations in advanced augmentation techniques (e.g., automated methods). Overall, our work rigorously contextualizes, both empirically and theoretically, the effects of data-centric properties on augmentation strategies and learning paradigms for graph SSL.

show abstract

“…Robinson et al [20] propose a way to select hard negative samples based on the embedding space distances, and use it to obtain high-quality graph embedding. There are also many works [21], [22] systemically studying the data augmentation on the graphs.…”

Section: Introductionmentioning

confidence: 99%

ARIEL: Adversarial Graph Contrastive Learning

Feng¹,

Jing²,

Zhu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Contrastive learning is an effective unsupervised method in graph representation learning, and the key component of contrastive learning lies in the construction of positive and negative samples. Previous methods usually utilize the proximity of nodes in the graph as the principle. Recently, the data augmentation based contrastive learning method has advanced to show great power in the visual domain, and some works extended this method from images to graphs. However, unlike the data augmentation on images, the data augmentation on graphs is far less intuitive and much harder to provide high-quality contrastive samples, which leaves much space for improvement. In this work, by introducing an adversarial graph view for data augmentation, we propose a simple but effective method, Adversarial Graph Contrastive Learning (ARIEL), to extract informative contrastive samples within reasonable constraints. We develop a new technique called information regularization for stable training and use subgraph sampling for scalability. We generalize our method from node-level contrastive learning to the graph-level by treating each graph instance as a supernode. ARIEL consistently outperforms the current graph contrastive learning methods for both node-level and graph-level classification tasks on real-world datasets. We further demonstrate that ARIEL is more robust in face of adversarial attacks.

show abstract

Data Augmentation for Graph Neural Networks

Cited by 139 publications

References 57 publications

Generic Structure Extraction with Bi-Level Optimization for Graph Structure Learning

Generic Structure Extraction with Bi-Level Optimization for Graph Structure Learning

Analyzing Data-Centric Properties for Graph Contrastive Learning

ARIEL: Adversarial Graph Contrastive Learning

Contact Info

Product

Resources

About