Sampling for Nyström Extension-Based Spectral Clustering

Abstract

Sampling is the key aspect for Nyström extension based spectral clustering. Traditional sampling schemes select the set of landmark points on a whole and focus on how to lower the matrix approximation error. However, the matrix approximation error does not have direct impact on the clustering performance. In this article, we propose a sampling framework from an incremental perspective, i.e., the landmark points are selected one by one, and each next point to be sampled is determined by previously selected landmark points. Incremental sampling builds explicit relationships among landmark points; thus, they work together well and provide a theoretical guarantee on the clustering performance. We provide two novel analysis methods and propose two schemes for selecting-the-next-one of the framework. The first scheme is based on clusterability analysis, which provides a better guarantee on clustering performance than schemes based on matrix approximation error analysis. The second scheme is based on loss analysis, which provides maximized predictive ability of the landmark points on the (implicit) labels of the unsampled points. Experimental results on a wide range of benchmark datasets demonstrate the superiorities of our proposed incremental sampling schemes over existing sampling schemes.

Fast Communication-Efficient Spectral Clustering over Distributed Data

“…Chen and Cai [15] proposed a landmark-based method for spectral clustering by selecting representative data points as a linear combination of the original data. Zhang and his co-authors [63] proposed an incremental sampling approach, i.e., the landmark points are selected one at a time adaptively based on the existing landmark points. Liu et al [37] proposed a fast constrained spectral clustering algorithm via landmark-based graph construction and then reduce the data size by random sampling after spectral embedding.…”

Multi-graph fusion for multi-view spectral clustering

“…The complexity of updating F is also O(n 3 ) due to the employment of SVD operation. To make our algorithm more efficient, several off-the-shell acceleration algorithms could be utilized, e.g., skinny SVD [61], sampling-based methods [62,63,64]. In our experiments, we don't apply these acceleration techniques.…”

“…sampling-based methods [62,63,64]. In our experiments, we don't apply these acceleration techniques.…”

Partial Sum Minimization of Singular Values Representation on Grassmann Manifolds

“…Clustering is a fundamental problem in computer vision and machine learning. A large number of methods have been proposed to solve this problem, such as the conventional iterative methods [23], [24], the statistical methods [25], [26], the factorization-based algebraic approaches [27]- [29], and the spectral clustering methods [4], [30]- [36]. Among all the clustering methods, the Spectral Clustering (SC) algorithm is state-of-the-art with excellent performance in many applications [37], [38] by exploring affinity information of data.…”