Rethinking Graph Regularization for Graph Neural Networks

Han, Yang; Ma, Kaili; Cheng, James

doi:10.48550/arxiv.2009.02027

Cited by 6 publications

(8 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It softens the onehot hard targets 𝑦 𝑐 = 1, 𝑦 𝑖 = 0 ∀𝑖 ≠ 𝑐 into 𝑦 𝐿𝑆 𝑖 = (1 − 𝛼)𝑦 𝑖 + 𝛼/𝐶, where 𝑐 is the correct label and 𝐶 is the number of classes. P-reg [44] is proposed as a variant of global graph Laplacian regularization to improve the GNNs with label information. It is defined as…”

Section: Resultsmentioning

confidence: 99%

“…Graph Laplacian regularization [2,51] helps to learn the node embeddings given the original graph. Han et al [44] propose…”

Section: Learning Graph Structure and Spreading Labelsmentioning

confidence: 99%

“…The results of applying deep GNNs are not desirable, but only being able to choose the number of layers 𝐿 from a discrete set N + is also not ideal. A balance between the size of the receptive field (the farthest nodes that a node can obtain information from) and the preservation of discriminating information is critical for generating useful node representation [44]. Thus, the regularization factor 𝛼 𝐿 ∈ R in Equation 10 is vital in enforcing a flexible strength of regularization on the GNN.…”

Section: Proofmentioning

confidence: 99%

See 2 more Smart Citations

LEReg: Empower Graph Neural Networks with Local Energy Regularization

Ma,

Chen,

Song

2022

Preprint

View full text Add to dashboard Cite

Researches on analyzing graphs with Graph Neural Networks (GNNs) have been receiving more and more attention because of the great expressive power of graphs. GNNs map the adjacency matrix and node features to node representations by message passing through edges on each convolution layer. However, the message passed through GNNs is not always beneficial for all parts in a graph. Specifically, as the data distribution is different over the graph, the receptive field (the farthest nodes that a node can obtain information from) needed to gather information is also different. Existing GNNs treat all parts of the graph uniformly, which makes it difficult to adaptively pass the most informative message for each unique part. To solve this problem, we propose two regularization terms that consider message passing locally: (1) Intra-Energy Reg and (2) Inter-Energy Reg. Through experiments and theoretical discussion, we first show that the speed of smoothing of different parts varies enormously and the topology of each part affects the way of smoothing. With Intra-Energy Reg, we strengthen the message passing within each part, which is beneficial for getting more useful information. With Inter-Energy Reg, we improve the ability of GNNs to distinguish different nodes. With the proposed two regularization terms, GNNs are able to filter the most useful information adaptively, learn more robustly and gain higher expressiveness. Moreover, the proposed LEReg can be easily applied to other GNN models with plug-and-play characteristics. Extensive experiments on several benchmarks verify that GNNs with LEReg outperform or match the state-of-the-art methods. The effectiveness and efficiency are also empirically visualized with elaborate experiments.

show abstract

Section: Resultsmentioning

confidence: 99%

“…Graph Laplacian regularization [2,51] helps to learn the node embeddings given the original graph. Han et al [44] propose…”

Section: Learning Graph Structure and Spreading Labelsmentioning

confidence: 99%

Section: Proofmentioning

confidence: 99%

See 1 more Smart Citation

LEReg: Empower Graph Neural Networks with Local Energy Regularization

Ma,

Chen,

Song

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Jiang et al [94] explore the way to do graph construction based on GCN. Yang et al [95] combine the classic graph regularization methods with GCN. Abu et al [96] present a novel N-GCN which marries the random walk with GCN, and a follow-up work GIL [97] with similar ideas is proposed as well.…”

Section: Generalized Aggregation Operationmentioning

confidence: 99%

Graph-based Semi-supervised Learning: A Comprehensive Review

Song¹,

Yang²,

Xu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Semi-supervised learning (SSL) has tremendous value in practice due to its ability to utilize both labeled data and unlabelled data. An important class of SSL methods is to naturally represent data as graphs such that the label information of unlabelled samples can be inferred from the graphs, which corresponds to graph-based semi-supervised learning (GSSL) methods. GSSL methods have demonstrated their advantages in various domains due to their uniqueness of structure, the universality of applications, and their scalability to large scale data. Focusing on this class of methods, this work aims to provide both researchers and practitioners with a solid and systematic understanding of relevant advances as well as the underlying connections among them. This makes our paper distinct from recent surveys that cover an overall picture of SSL methods while neglecting fundamental understanding of GSSL methods. In particular, a major contribution of this paper lies in a new generalized taxonomy for GSSL, including graph regularization and graph embedding methods, with the most up-to-date references and useful resources such as codes, datasets, and applications. Furthermore, we present several potential research directions as future work with insights into this rapidly growing field.

show abstract

“…Some have suggested to constrain the parameter space of GNN so that the trained GNN becomes a non-expansive map, thus producing the fixed point [17,36]. Others have proposed to apply an additional GNN layer on the embedded graph and penalize the difference between the output of the additional GNN layer and the embedded graph to guide the GNN to find the fixed points [41]. It has been shown that regularizing GNN to find its fixed points improves the predictive performance of GNN [36,41].…”

Section: Related Workmentioning

confidence: 99%

Convergent Graph Solvers

Park¹,

Choo²,

Park³

2021

Preprint

View full text Add to dashboard Cite

We propose the convergent graph solver (CGS) 1 , a deep learning method that learns iterative mappings to predict the properties of a graph system at its stationary state (fixed point) with guaranteed convergence. CGS systematically computes the fixed points of a target graph system and decodes them to estimate the stationary properties of the system without the prior knowledge of existing solvers or intermediate solutions. The forward propagation of CGS proceeds in three steps: (1) constructing the input dependent linear contracting iterative maps, (2) computing the fixed-points of the linear maps, and (3) decoding the fixed-points to estimate the properties. The contractivity of the constructed linear maps guarantees the existence and uniqueness of the fixed points following the Banach fixed point theorem. To train CGS efficiently, we also derive a tractable analytical expression for its gradient by leveraging the implicit function theorem. We evaluate the performance of CGS by applying it to various network-analytic and graph benchmark problems. The results indicate that CGS has competitive capabilities for predicting the stationary properties of graph systems, irrespective of whether the target systems are linear or non-linear. CGS also shows high performance for graph classification problems where the existence or the meaning of a fixed point is hard to be clearly defined, which highlights the potential of CGS as a general graph neural network architecture.

show abstract

Rethinking Graph Regularization for Graph Neural Networks

Cited by 6 publications

References 23 publications

LEReg: Empower Graph Neural Networks with Local Energy Regularization

LEReg: Empower Graph Neural Networks with Local Energy Regularization

Graph-based Semi-supervised Learning: A Comprehensive Review

Convergent Graph Solvers

Contact Info

Product

Resources

About