Coherence Pursuit: Fast, Simple, and Robust Principal Component Analysis

Abstract

This paper presents a remarkably simple, yet powerful, algorithm termed Coherence Pursuit (CoP) to robust Principal Component Analysis (PCA). As inliers lie in a low dimensional subspace and are mostly correlated, an inlier is likely to have strong mutual coherence with a large number of data points. By contrast, outliers either do not admit low dimensional structures or form small clusters. In either case, an outlier is unlikely to bear strong resemblance to a large number of data points. Given that, CoP sets an outlier apart from an inlier by comparing their coherence with the rest of the data points. The mutual coherences are computed by forming the Gram matrix of the normalized data points. Subsequently, the sought subspace is recovered from the span of the subset of the data points that exhibit strong coherence with the rest of the data. As CoP only involves one simple matrix multiplication, it is significantly faster than the state-of-the-art robust PCA algorithms. We derive analytical performance guarantees for CoP under different models for the distributions of inliers and outliers in both noise-free and noisy settings. CoP is the first robust PCA algorithm that is simultaneously non-iterative, provably robust to both unstructured and structured outliers, and can tolerate a large number of unstructured outliers

A Robust Principal Component Analysis for Outlier Identification in Messy Microcalorimeter Data

“…The method best suited to microcalorimeter analysis is known as coherence pursuit. 28 It relies on the idea that good columns in M (here, clean pulses) will tend to lie in nearly the same direction in R m as many other columns, while outlier columns will tend to lie far from all or most other columns, even in the extreme case that outliers outnumber the clean pulses. The underlying idea that random high-dimensional points are nearly always orthogonal to one another can be made precise.…”

Dual Principal Component Pursuit

“…The recent work of Rahmani and Atia (2017) analyzes a simple yet efficient algorithm for detecting the inlier space from pairwise point coherences, hence called Coherence Pursuit (CoP). The main insight behind CoP is that, under the hypothesis that the inliers lie in a low-dimensional subspace, inlier points tend to have significantly higher coherence with the rest of the points in the dataset, than outlier points.…”

“…Similarly to SE-RPCA, the performance of CoP is not expected to be significantly degraded as the number of outliers increases, as long as each outlier remains sufficiently incoherent with the rest of the dataset. As demonstrated by Rahmani and Atia (2017), CoP has a competitive performance and admits an extensive theoretical analysis.…”

“…In particular, we test DPCP-LP (Algorithm 1), DPCP-IRLS (Algorithm 2), DPCP-d (Algorithm 3), RANSAC (Fischler and Bolles, 1981), SE-RPCA (Soltanolkotabi and Candès, 2012), 21 -RPCA (Xu et al, 2012), the IRLS version of REAPER (Lerman et al, 2015), as well as Coherence Pursuit (CoP) (Rahmani and Atia, 2017); see §2 for details on these last five existing methods. whenn 0 is initialized from the SVD of the data or uniformly at random, respectively.…”

Compressed Randomized Utv Decompositions for Low-rank Matrix Approximations in Data Science

“…Such compact representations which retain the key features of a high-dimensional matrix provide a significant reduction in memory requirements, and more importantly, computational costs when the latter scales, e.g., according to a high-degree polynomial, with the dimensionality. Matrices with low-rank structures have found many applications in background subtraction [1,2], system identification [3], IP network anomaly detection [4,5], latent variable graphical modeling [6], subspace clustering [7,8] and sensor and multichannel signal processing [9,10,11,12,13,14,15], [16,17,18,19,20,21,22,23,24,25,26,27]. .…”