Fast Robust PCA on Graphs

Abstract

Mining useful clusters from high dimensional data has received significant attention of the computer vision and pattern recognition community in the recent years. Linear and non-linear dimensionality reduction has played an important role to overcome the curse of dimensionality. However, often such methods are accompanied with three different problems: high computational complexity (usually associated with the nuclear norm minimization), non-convexity (for matrix factorization methods) and susceptibility to gross corruptions in the data. In this paper we propose a principal component analysis (PCA) based solution that overcomes these three issues and approximates a low-rank recovery method for high dimensional datasets. We target the low-rank recovery by enforcing two types of graph smoothness assumptions, one on the data samples and the other on the features by designing a convex optimization problem. The resulting algorithm is fast, efficient and scalable for huge datasets with O(nlog(n)) computational complexity in the number of data samples. It is also robust to gross corruptions in the dataset as well as to the model parameters. Clustering experiments on 7 benchmark datasets with different types of corruptions and background separation experiments on 3 video datasets show that our proposed model outperforms 10 state-of-the-art dimensionality reduction models. Our theoretical analysis proves that the proposed model is able to recover approximate low-rank representations with a bounded error for clusterable data

Gaussian Processes Over Graphs

“…This includes signal analysis concepts such as Fourier transforms [32], filtering [30], [33], wavelets [34], [35], filterbanks [36], [37], multiresolution analysis [38]- [40], denoising [41], [42], and dictionary learning [43], [44], and stationary signal analysis [45], [46]. Spectral clustering and principal component analysis approaches based on graph signal filtering have also been proposed recently [47], [48]. Several approaches have been proposed for learning the graph directly from data [49], [50].…”

Data-Driven Tree Transforms and Metrics

“…This work is also related to the matrix factorization proposed by Shahid et al [22], where the graph Laplacians of both the features and the observation regularize the decomposition of a dataset into a low-rank matrix and a sparse matrix representing noise. Then the observations are clustered using k-means on the low-dimensional principal components of the smooth low-rank matrix.…”

Side Information in Robust Principal Component Analysis: Algorithms and Applications

“…(3) [23,24] incorporate structural knowledge into RPCA by adding spectral graph regularisation. Given the graph Laplacian Φ of each data similarity graph, Robust PCA on Graphs (RPCAG) and Fast Robust PCA on Graphs (FRPCAG) add an additional tr(LΦL T ) term to the PCP objective for the low-rank component L. The main drawback of the above mentioned models is that the side information needs to be accurate and noiseless, which is not trivial in practical scenarios.…”

“…The tolerance threshold for RPCAG and FR-PCAG are all set to = 10 −7 for reasons of consistency. We choose λ = 1/ max(n 1 , n 2 ) for a general matrix of dimension n 1 × n 2 as suggested in [23,24]. For simulation experiments, γ in RPCAG is given by the minimiser (at γ = 0.2) of L−L0 F L0 F on the benchmark problem (Figure 7).…”