Restricted Isometry Property of Gaussian Random Projection for Finite Set of Subspaces

Abstract

Dimension reduction plays an essential role when decreasing the complexity of solving large-scale problems. The well-known Johnson-Lindenstrauss (JL) Lemma and Restricted Isometry Property (RIP) admit the use of random projection to reduce the dimension while keeping the Euclidean distance, which leads to the boom of Compressed Sensing and the field of sparsity related signal processing. Recently, successful applications of sparse models in computer vision and machine learning have increasingly hinted that the underlying structure of high dimensional data looks more like a union of subspaces (UoS). In this paper, motivated by JL Lemma and an emerging field of Compressed Subspace Clustering (CSC), we study for the first time the RIP of Gaussian random matrices for the compression of two subspaces based on the generalized projection F -norm distance. We theoretically prove that with high probability the affinity or distance between two projected subspaces are concentrated around their estimates. When the ambient dimension after projection is sufficiently large, the affinity and distance between two subspaces almost remain unchanged after random projection. Numerical experiments verify the theoretical work.

Rigorous restricted isometry property of low-dimensional subspaces

“…In order to give more solid guarantees and more precise insight into the law of magnitude of the dimensions for CSC and other subspace related problems, we derive an optimum probability bound of the RIP of Gaussian random compressions for subspaces in this paper. Compared with our previous work [21], the probability bound has been improve from 1 − O(1/n) to 1 − e −O(n) , which is optimum when we consider the state-of-the-art statistical probability theories for Gaussian random matrix.…”

“…It should be noted that we slightly generalize the definition in [22] to the situation where the dimensions of the two subspaces are different. Definition 1 ( [21]) (Projection Frobenius norm distance between subspaces) The generalized projection F-norm distance between two subspaces X 1 and X 2 is defined as…”

“…Definition 3 [11,12,13] The projection matrix Φ ∈ R n×N , n < N satisfies RIP of order k if there exists a δ k ∈ (0, 1) such that Table 1: Comparison with other dimension-reduction theories including JL Lemma, RIP for sparse signals, and our previous results [21] RIP for low-dimensional subspaces JL Lemma RIP for sparse signals [21] this work…”

“…Except our previous work [21] that will be compared with in Section 6, as far as we know, there is no relevant work that study the distance preserving property between subspaces after random projection.…”

“…Our theoretical analysis will first focus on the estimation of these quantities before and after random projection. Then using the connection between affinity and projection F-norm distance derived in [21], we can readily derive the result in Theorem 1.…”