Oracle Based Active Set Algorithm for Scalable Elastic Net Subspace Clustering

You, Chong; Li, Chunguang; Robinson, Daniel N.; Vidal, René

doi:10.1109/cvpr.2016.426

Cited by 211 publications

(180 citation statements)

References 37 publications

Supporting

Mentioning

169

Contrasting

Order By: Relevance

“…It was reported in [48] that for this data set, even using just two iterations of the solvers, that OMP took 783 minutes, and two SSC-1 methods (one based on the original SSC ADMM algorithm, without using Remark 1) either did not finish the two iterations within 7 days, or used more than 16 GB of memory. The results of running our models on this data (after normalizing the features to z-scores), and taking 2 steps as in [48], is presented in Table 2. We do not include results for SSC-1 with affine constraints, as with default parameters this model does not lead to a sparse C, and hence there are memory issues.…”

Section: Ssc-1 and Ssc-0 On The Covertype Data Setmentioning

confidence: 96%

Efficient Solvers for Sparse Subspace Clustering

2020

View full text Add to dashboard Cite

Sparse subspace clustering (SSC) clusters n points that lie near a union of low-dimensional subspaces. The SSC model expresses each point as a linear or affine combination of the other points, using either 1 or 0 regularization. Using 1 regularization results in a convex problem but requires O(n 2 ) storage, and is typically solved by the alternating direction method of multipliers which takes O(n 3 ) flops. The 0 model is non-convex but only needs memory linear in n, and is solved via orthogonal matching pursuit and cannot handle the case of affine subspaces. This paper shows that a proximal gradient framework can solve SSC, covering both 1 and 0 models, and both linear and affine constraints. For both 1 and 0 , algorithms to compute the proximity operator in the presence of affine constraints have not been presented in the SSC literature, so we derive an exact and efficient algorithm that solves the 1 case with just O(n 2 ) flops. In the 0 case, our algorithm retains the low-memory overhead, and is the first algorithm to solve the SSC-0 model with affine constraints. Experiments show our algorithms do not rely on sensitive regularization parameters, and they are less sensitive to sparsity misspecification and high noise. * This paper is accepted for publication in Signal Processing.

show abstract

Section: Ssc-1 and Ssc-0 On The Covertype Data Setmentioning

confidence: 96%

Efficient Solvers for Sparse Subspace Clustering

2020

View full text Add to dashboard Cite

show abstract

“…In this section, the performance of SR-SSC is evaluated using three large-scale and challenging data sets: handwritten digits (MNIST), object images (CIFAR10) and forest data set (Covertype). We compare the accuracy and running times of SR-SSC with standard SSC based on ADMM (this is SR-SSC with L = 1 and k = N ) and three state-of-the art methods for sparse subspace clustering, namely OMP [40], SSSC [26] (which is closely related to SR-SSC with L = 1) and ORGEN [39]; see Section 2 for a brief description of these methods. For OMP and ORGEN, we used the code available from http://vision.jhu.edu/code/.…”

Section: Real-world Data Setsmentioning

confidence: 99%

Scalable and robust sparse subspace clustering using randomized clustering and multilayer graphs

2019

View full text Add to dashboard Cite

Sparse subspace clustering (SSC) is one of the current state-of-the-art methods for partitioning data points into the union of subspaces, with strong theoretical guarantees. However, it is not practical for large data sets as it requires solving a LASSO problem for each data point, where the number of variables in each LASSO problem is the number of data points. To improve the scalability of SSC, we propose to select a few sets of anchor points using a randomized hierarchical clustering method, and, for each set of anchor points, solve the LASSO problems for each data point allowing only anchor points to have a non-zero weight (this reduces drastically the number of variables). This generates a multilayer graph where each layer corresponds to a different set of anchor points. Using the Grassmann manifold of orthogonal matrices, the shared connectivity among the layers is summarized within a single subspace. Finally, we use k-means clustering within that subspace to cluster the data points, similarly as done by spectral clustering in SSC. We show on both synthetic and real-world data sets that the proposed method not only allows SSC to scale to large-scale data sets, but that it is also much more robust as it performs significantly better on noisy data and on data with close susbspaces and outliers, while it is not prone to oversegmentation.

show abstract

“…regularization they choose for sparse coefficients. For instance, Low Rank Representation (LRR) [7,8] based on nuclear norm represents the data points with the lowest-rank representation among all the candidates; Least Square Regression (LSR) [9] with 2 norm groups the highly correlated data; Exemplar-based Subspace Clustering (ESC) [10] based on 1 norm focuses on the class-imbalanced data; Elastic Net Subspace Clustering (ENSC) [11] adopts both 1 and 2 norm to find better coefficients; Subspace Learning by 0 -Induced Sparisty [12] employs proximal gradient descent to obtain a sub-optimal solution; while Sparse Subspace Clustering (SSC) [13] with the 1 norm calculates a sparse self-representation of data points. And Structured Sparse Clustering [14] learns both affinity and segmentation of SSC.…”

Section: Introductionmentioning

confidence: 99%

Restricted Connection Orthogonal Matching Pursuit for Sparse Subspace Clustering

Zhu

Yang

Zhu

2019

IEEE Signal Process. Lett.

View full text Add to dashboard Cite

Sparse Subspace Clustering (SSC) is one of the most popular methods for clustering data points into their underlying subspaces. However, SSC may suffer from heavy computational burden. Orthogonal Matching Pursuit applied on SSC accelerates the computation but the trade-off is the loss of clustering accuracy. In this paper, we propose a noise-robust algorithm, Restricted Connection Orthogonal Matching Pursuit for Sparse Subspace Clustering (RCOMP-SSC), to improve the clustering accuracy and maintain the low computational time by restricting the number of connections of each data point during the iteration of OMP. Also, we develop a framework of control matrix to realize RCOMP-SCC. And the framework is scalable for other data point selection strategies. Our analysis and experiments on synthetic data and two real-world databases (EYaleB & Usps) demonstrate the superiority of our algorithm compared with other clustering methods in terms of accuracy and computational time.

show abstract

Oracle Based Active Set Algorithm for Scalable Elastic Net Subspace Clustering

Abstract: Abstract

Cited by 211 publications

References 37 publications

Efficient Solvers for Sparse Subspace Clustering

Efficient Solvers for Sparse Subspace Clustering

Scalable and robust sparse subspace clustering using randomized clustering and multilayer graphs

Restricted Connection Orthogonal Matching Pursuit for Sparse Subspace Clustering

Contact Info

Product

Resources

About