Efficient computation of node proximity queries such as transition probabilities, Personalized PageRank, and Katz are of fundamental importance in various graph mining and learning tasks. In particular, several recent works leverage fast node proximity computation to improve the scalability of Graph Neural Networks (GNN). However, prior studies on proximity computation and GNN feature propagation are on a case-by-case basis, with each paper focusing on a particular proximity measure.In this paper, we propose Approximate Graph Propagation (AGP), a unified randomized algorithm that computes various proximity queries and GNN feature propagation, including transition probabilities, Personalized PageRank, heat kernel PageRank, Katz, SGC, GDC, and APPNP. Our algorithm provides a theoretical bounded error guarantee and runs in almost optimal time complexity. We conduct an extensive experimental study to demonstrate AGP's effectiveness in two concrete applications: local clustering with heat kernel PageRank and node classification with GNNs. Most notably, we present an empirical study on a billion-edge graph Papers100M, the largest publicly available GNN dataset so far. The results show that AGP can significantly improve various existing GNN models' scalability without sacrificing prediction accuracy.
Designing spectral convolutional networks is a challenging problem in graph learning. ChebNet, one of the early attempts, approximates the spectral convolution using Chebyshev polynomials. GCN simplifies ChebNet by utilizing only the first two Chebyshev polynomials while still outperforming it on real-world datasets. GPR-GNN and BernNet demonstrate that the Monomial and Bernstein bases also outperform the Chebyshev basis in terms of learning the spectral convolution. Such conclusions are counter-intuitive in the field of approximation theory, where it is established that the Chebyshev polynomial achieves the optimum convergent rate for approximating a function. In this paper, we revisit the problem of approximating the spectral convolution with Chebyshev polynomials. We show that ChebNet's inferior performance is primarily due to illegal coefficients learnt by ChebNet approximating analytic filter functions, which leads to over-fitting. We then propose ChebNetII, a new GNN model based on Chebyshev interpolation, which enhances the original Chebyshev polynomial approximation while reducing the Runge phenomena. We conducted an extensive experimental study to demonstrate that ChebNetII can learn arbitrary graph spectrum filters and achieve superior performance in both full-and semi-supervised node classification tasks. 1 IntroductionGraph neural networks (GNNs) have received considerable attention in recent years due to their remarkable performance on a variety of graph learning tasks, including social analysis [27,21,33], drug discovery [43,16,28], traffic forecasting [22,3,7], recommendation system [36,40] and computer vision [42,5].Spatial-based and spectral-based graph neural networks (GNNs) are the two primary categories of GNNs. To learn node representations, spatial-based GNNs [18,13,34] often rely on a message propagation and aggregation mechanism between neighboring nodes. Spectral-based methods [8] create spectrum graph convolutions or, equivalently, spectral graph filters, in the spectral domain of the Laplacian matrix. We can further divide spectral-based GNNs into two categories based on whether or not their graph convolutions can be learned.• Predetermined graph convolutions: GCN [18] uses a simplified first-order Chebyshev polynomial as graph convolution, which is proven to be a low-pass filter [1,35,37,44]. APPNP [19] utilizes Personalized PageRank (PPR) to set the graph convolution and achieves a low-pass filter as well [20,44]. GNN-LF/HF [44] designs graph convolutions from the perspective of graph optimization functions, which can simulate high-and low-pass filters. • Learnable graph convolutions: ChebNet [8] approximates the graph convolution with Chebyshev polynomials and learns the convolutional filters via trainable weights of the Chebyshev basis. GPR-GNN [6] uses the Monomial basis to approximate graph convolutions, which can derive high-or low-pass filters. ARMA [2] learns a rational convolutional filter via the family of Auto-Regressive Moving Average filters [24]. BernNet [15] utilizes th...
Many representative graph neural networks, e.g., GPR-GNN and ChebyNet, approximate graph convolutions with graph spectral filters. However, existing work either applies predefined filter weights or learns them without necessary constraints, which may lead to oversimplified or ill-posed filters. To overcome these issues, we propose BernNet, a novel graph neural network with theoretical support that provides a simple but effective scheme for designing and learning arbitrary graph spectral filters. In particular, for any filter over the normalized Laplacian spectrum of a graph, our BernNet estimates it by an order-K Bernstein polynomial approximation and designs its spectral property by setting the coefficients of the Bernstein basis. Moreover, we can learn the coefficients (and the corresponding filter weights) based on observed graphs and their associated signals and thus achieve the BernNet specialized for the data. Our experiments demonstrate that BernNet can learn arbitrary spectral filters, including complicated band-rejection and comb filters, and it achieves superior performance in real-world graph modeling tasks. K k=0 w k L k x, w k 's are the filter weights, L = I − D −1/2 AD −1/2 is the symmetric normalized Laplacian matrix of G, and D is the diagonal degree matrix of A. Another equivalent polynomial filtering operation is K k=0 c k P k x, where P = D −1/2 AD −1/2 is the normalized adjacency matrix and c k 's are the filter weights.We can broadly categorize the GNNs applying the above filtering operation into two classes, depending on whether they design the filter weights or learn them based on observed graphs. Some representative models in these two classes are shown below.• The GNNs driven by designing filters: GCN [12] uses a simplified first-order Chebyshev polynomial, which is proven to be a low-pass filter [1,29,32,39]. APPNP [13] utilizes Personalized PageRank (PPR) to set the filter weights and achieves a low-pass filter as well [14,39]. GNN-LF/HF [39] designs filter weights from the perspective of graph optimization functions, which can simulate high-and low-pass filters.Preprint. Under review.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.