Learning Sparse Graph Laplacian with K Eigenvector Prior via Iterative Glasso and Projection

Bagheri, Saghar; Cheung, Gene; Ortega, Antonio; Fen, Wang

doi:10.1109/icassp39728.2021.9414693

Cited by 4 publications

(16 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It can be easily proven that C, U ≥ 0 [25]. Thus, the last eigen-pair of C is (λ N , v N ) = ( C, U , u).…”

Section: A Computing Last Eigen-pair (λ N U)mentioning

confidence: 94%

“…The constraints require eigenvectors of a real symmetric matrix C to be orthonormal. Objective in ( 4) is equivalent to tr(v ⊤ E N −1 v), which is quadratic and convex, given E N −1 can be proven to be PSD [25]. Thus, the maximization ( 4) is non-convex and NP-hard.…”

Section: B Computing Next Eigen-pairmentioning

confidence: 99%

“…The previous eigen-component computation constitutes a projection C = Proj( C)-one can prove this operator is indeed idempotent, i.e., Proj(Proj( C)) = Proj( C) [25]. Next, we formulate the following GLASSO-like optimization problem to estimate a graph Laplacian matrix L [31]:…”

Section: Glasso Formulationmentioning

confidence: 99%

“…We solve (13) iteratively using our projection operator Proj(•) and a variant of the block Coordinate descent (BCD) algorithm in [32]. Specifically, similarly done in [22], we solve the dual of GLASSO as follows. Note first that the l 1 -norm in ( 13) can be written as…”

Section: Glasso Formulationmentioning

confidence: 99%

“…In this paper, leveraging a recent spectral graph learning scheme [22], we learn a sparse graph from empirical data with a tunable eigen-gap, in order to promote expressiveness in the trained GCN and optimize performance. Unlike works [7], [8] that indirectly induced eigen-gaps by sparsifying graphs heuristically in the nodal domain, we directly engineer an eigen-gap by optimizing one eigen-pair of the graph Laplacian at a time in the spectral domain.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Sparse Graph Learning with Eigen-gap for Spectral Filter Training in Graph Convolutional Networks

Zeng¹,

Bagheri²,

Liu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

It is now known that the expressive power of graph convolutional neural nets (GCN) does not grow infinitely with the number of layers. Instead, the GCN output approaches a subspace spanned by the first eigenvector of the normalized graph Laplacian matrix with the convergence rate characterized by the "eigen-gap": the difference between the Laplacian's first two distinct eigenvalues. To promote a deeper GCN architecture with sufficient expressiveness, in this paper, given an empirical covariance matrix C computed from observable data, we learn a sparse graph Laplacian matrix L closest to C−1 while maintaining a desirable eigen-gap that slows down convergence. Specifically, we first define a sparse graph learning problem with constraints on the first eigenvector (the most common signal) and the eigen-gap. We solve the corresponding dual problem greedily, where a locally optimal eigen-pair is computed one at a time via a fast approximation of a semi-definite programming (SDP) formulation. The computed L with the desired eigen-gap is normalized spectrally and used for supervised training of GCN for a targeted task. Experiments show that our proposal produced deeper GCNs and smaller errors compared to a competing scheme without explicit eigen-gap optimization.

show abstract

“…It can be easily proven that C, U ≥ 0 [25]. Thus, the last eigen-pair of C is (λ N , v N ) = ( C, U , u).…”

Section: A Computing Last Eigen-pair (λ N U)mentioning

confidence: 94%

Section: B Computing Next Eigen-pairmentioning

confidence: 99%

Section: Glasso Formulationmentioning

confidence: 99%

Section: Glasso Formulationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Sparse Graph Learning with Eigen-gap for Spectral Filter Training in Graph Convolutional Networks

Zeng¹,

Bagheri²,

Liu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Sparse Graph Learning with Spectrum Prior for Deep Graph Convolutional Networks

Zeng

Liu

Cheung

et al. 2023

ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Hybrid Model-Based / Data-Driven Graph Transform for Image Coding

Bagheri

Cheung

et al. 2022

2022 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

Transform coding to sparsify signal representations remains crucial in an image compression pipeline. While the Karhunen-Loève transform (KLT) computed from an empirical covariance matrix C is theoretically optimal for a stationary process, in practice, collecting sufficient statistics from a non-stationary image to reliably estimate C can be difficult. In this paper, to encode an intra-prediction residual block, we pursue a hybrid model-based / data-driven approach: the first K eigenvectors of a transform matrix are derived from a statistical model, e.g., the asymmetric discrete sine transform (ADST), for stability, while the remaining N − K are computed from C for data adaptivity. The transform computation is posed as a graph learning problem, where we seek a graph Laplacian matrix minimizing a graphical lasso objective inside a convex cone sharing the first K eigenvectors in a Hilbert space of real symmetric matrices. We efficiently solve the problem via augmented Lagrangian relaxation and proximal gradient (PG). Using open-source WebP as a baseline image codec, experimental results show that our hybrid graph transform achieved better coding performance than discrete cosine transform (DCT), ADST and KLT, and better stability than KLT.

show abstract

Learning Sparse Graph Laplacian with K Eigenvector Prior via Iterative Glasso and Projection

Cited by 4 publications

References 24 publications

Sparse Graph Learning with Eigen-gap for Spectral Filter Training in Graph Convolutional Networks

Sparse Graph Learning with Eigen-gap for Spectral Filter Training in Graph Convolutional Networks

Sparse Graph Learning with Spectrum Prior for Deep Graph Convolutional Networks

Hybrid Model-Based / Data-Driven Graph Transform for Image Coding

Contact Info

Product

Resources

About