Graph Signal Processing -- Part II: Processing and Analyzing Signals on Graphs

Stankovic, Ljubisa; Mandic, Danilo; Dakovic, Milos; Brajovic, Milos; Scalzo, Bruno; Constantinides, Anthony G.

doi:10.48550/arxiv.1909.10325

Cited by 7 publications

(13 citation statements)

References 66 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Chebyshev basis vs. other bases. Chebyshev polynomials are widely used to approximate various functions in digital signal processing and graph signal filtering [31,32]. It has been shown that for analytic functions in an ellipse containing the approximation interval, the truncated Chebyshev expansions give an approximate minimax polynomial [11].…”

Section: The Motivation Of Revisiting Chebnetmentioning

confidence: 99%

Convolutional Neural Networks on Graphs with Chebyshev Approximation, Revisited

He¹,

Wei²,

Wen³

2022

Preprint

View full text Add to dashboard Cite

Designing spectral convolutional networks is a challenging problem in graph learning. ChebNet, one of the early attempts, approximates the spectral convolution using Chebyshev polynomials. GCN simplifies ChebNet by utilizing only the first two Chebyshev polynomials while still outperforming it on real-world datasets. GPR-GNN and BernNet demonstrate that the Monomial and Bernstein bases also outperform the Chebyshev basis in terms of learning the spectral convolution. Such conclusions are counter-intuitive in the field of approximation theory, where it is established that the Chebyshev polynomial achieves the optimum convergent rate for approximating a function. In this paper, we revisit the problem of approximating the spectral convolution with Chebyshev polynomials. We show that ChebNet's inferior performance is primarily due to illegal coefficients learnt by ChebNet approximating analytic filter functions, which leads to over-fitting. We then propose ChebNetII, a new GNN model based on Chebyshev interpolation, which enhances the original Chebyshev polynomial approximation while reducing the Runge phenomena. We conducted an extensive experimental study to demonstrate that ChebNetII can learn arbitrary graph spectrum filters and achieve superior performance in both full-and semi-supervised node classification tasks. 1 IntroductionGraph neural networks (GNNs) have received considerable attention in recent years due to their remarkable performance on a variety of graph learning tasks, including social analysis [27,21,33], drug discovery [43,16,28], traffic forecasting [22,3,7], recommendation system [36,40] and computer vision [42,5].Spatial-based and spectral-based graph neural networks (GNNs) are the two primary categories of GNNs. To learn node representations, spatial-based GNNs [18,13,34] often rely on a message propagation and aggregation mechanism between neighboring nodes. Spectral-based methods [8] create spectrum graph convolutions or, equivalently, spectral graph filters, in the spectral domain of the Laplacian matrix. We can further divide spectral-based GNNs into two categories based on whether or not their graph convolutions can be learned.• Predetermined graph convolutions: GCN [18] uses a simplified first-order Chebyshev polynomial as graph convolution, which is proven to be a low-pass filter [1,35,37,44]. APPNP [19] utilizes Personalized PageRank (PPR) to set the graph convolution and achieves a low-pass filter as well [20,44]. GNN-LF/HF [44] designs graph convolutions from the perspective of graph optimization functions, which can simulate high-and low-pass filters. • Learnable graph convolutions: ChebNet [8] approximates the graph convolution with Chebyshev polynomials and learns the convolutional filters via trainable weights of the Chebyshev basis. GPR-GNN [6] uses the Monomial basis to approximate graph convolutions, which can derive high-or low-pass filters. ARMA [2] learns a rational convolutional filter via the family of Auto-Regressive Moving Average filters [24]. BernNet [15] utilizes th...

show abstract

Section: The Motivation Of Revisiting Chebnetmentioning

confidence: 99%

Convolutional Neural Networks on Graphs with Chebyshev Approximation, Revisited

He¹,

Wei²,

Wen³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…where l represents the index of each layer, c l is the number of filters (channels) of the l-th layer, Θ l i,j is a diagonal matrix which contains the set of learnt parameters of the l-th layer, and ρ(•) is the activation function of neurons. In (121), the summation ensures the aggregation of features filtered by different convolutional kernels, Θ l i,j , which is similar to a linear combination across kernels in CNNs. Although it achieves graph convolution through NNs, this work has two main limitations: i) the localisation at the vertex domain cannot be ensured by Θ l i,j , although it is crucial in convolutional neural networks to extract local stationary features; ii) computational complexity brought by the O(N 2 ) multiplications of U and U T , and the eigendecomposition of L to obtain U, at the first time, may be prohibitive for large graphs.…”

Section: Graph Fourier Transformmentioning

confidence: 99%

“…the volume normalized cut, CutV (V 1 , V 2 ) in (156), takes the form of a generalised Rayleigh quotient of L, given by [120,121] CutV…”

Section: Spectral Bisection Based Minimum Cutmentioning

confidence: 99%

Graph Signal Processing -- Part III: Machine Learning on Graphs, from Graph Topology to Applications

Stanković¹,

Mandic²,

Daković³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Many modern data analytics applications on graphs operate on domains where graph topology is not known a priori, and hence its determination becomes part of the problem definition, rather than serving as prior knowledge which aids the problem solution. Part III of this monograph starts by addressing ways to learn graph topology, from the case where the physics of the problem already suggest a possible topology, through to most general cases where the graph topology is learned from the data. A particular emphasis is on graph topology definition based on the correlation and precision matrices of the observed data, combined with additional prior knowledge and structural conditions, such as the smoothness or sparsity of graph connections. For learning sparse graphs (with small number of edges), the least absolute shrinkage and selection operator, known as LASSO is employed, along with its graph specific variant, graphical LASSO. For completeness, both variants of LASSO are derived in an intuitive way, and explained. An in-depth elaboration of the graph topology learning paradigm is provided through several examples on physically well defined graphs, such as electric circuits, linear heat transfer, social and computer networks, and spring-mass systems. As many graph neural networks (GNN) and convolutional graph networks (GCN) are emerging, we have also reviewed the main trends in GNNs and GCNs, from the perspective of graph signal filtering. We have in particular studied the diffusion process over graphs and have shown that the trend of various improvements on GCNs can also be understood from the graph diffusion perspective. Given that the existing GCNs have been introduced largely in a heuristic manner, the definition of different diffusion processes can also serve as a basis for a new design of GCNs. Tensor representation of lattice-structured graphs is next considered, and it is shown that tensors (multidimensional data arrays) are a special class of graph signals, whereby the graph vertices reside on a high-dimensional regular lattice structure. This part of monograph concludes with two emerging applications in financial data processing and underground transportation networks modeling. By means of portfolio cuts of an asset graph, we show how domain knowledge can be meaningfully incorporated into investment analysis. In the underground traffic example, we demonstrate how graph theory can be used to identify the stations in the London underground network which have the greatest influence on the functionality of the traffic, and proceed, in an innovative way, to assess the impact of a station closure on service levels across the city. Contents 1 Introduction 2 2 Geometrically Defined Graph Topologies 3 3 Graph Topology Based on Signal Similarity 4 4 Learning of Graph Laplacian from Data 8 4.1 Imposing Sparsity on the Connection Metric 10 4.2 Smoothness Constrained Learning of Graph Laplacian .

show abstract

“…Graph Shift Filters The weighted adjacency matrix can be used as a shift operator to filter signals on graphs. Such a graph filter represents a linear combination of vertex-shifted graph signals, which captures graph information at a local level [15]. For example, the operation g = (I + A)f produces a filtered signal, g ∈ R N , such that g n = f n + m∈Ωn a n,m f m , where Ω n denotes the 1-hop neighbours that are directly connected to the n-th node.…”

Section: Graph Signal Processingmentioning

confidence: 99%

“…For example, the operation g = (I + A)f produces a filtered signal, g ∈ R N , such that g n = f n + m∈Ωn a n,m f m , where Ω n denotes the 1-hop neighbours that are directly connected to the n-th node. For M graph signals stacked in a matrix form as F ∈ R N ×M , the resulting graph filter can be compactly written as G = (I + A)F [15].…”

Section: Graph Signal Processingmentioning

confidence: 99%

Multi-Graph Tensor Networks

Xu,

Konstantinidis,

Mandic

2020

Preprint

View full text Add to dashboard Cite

The irregular and multi-modal nature of numerous modern data sources poses serious challenges for traditional deep learning algorithms. To this end, recent efforts have generalized existing algorithms to irregular domains through graphs, with the aim to gain additional insights from data through the underlying graph topology. At the same time, tensor-based methods have demonstrated promising results in bypassing the bottlenecks imposed by the Curse of Dimensionality. In this paper, we introduce a novel Multi-Graph Tensor Network (MGTN) framework, which exploits both the ability of graphs to handle irregular data sources and the compression properties of tensor networks in a deep learning setting. The potential of the proposed framework is demonstrated through an MGTN based deep Q agent for Foreign Exchange (FOREX) algorithmic trading. By virtue of the MGTN, a FOREX currency graph is leveraged to impose an economically meaningful structure on this demanding task, resulting in a highly superior performance against three competing models and at a drastically lower complexity.Preprint. Under review.

show abstract

Graph Signal Processing -- Part II: Processing and Analyzing Signals on Graphs

Cited by 7 publications

References 66 publications

Convolutional Neural Networks on Graphs with Chebyshev Approximation, Revisited

Convolutional Neural Networks on Graphs with Chebyshev Approximation, Revisited

Graph Signal Processing -- Part III: Machine Learning on Graphs, from Graph Topology to Applications

Multi-Graph Tensor Networks

Contact Info

Product

Resources

About