Improved Spectral-Norm Bounds for Clustering

Abstract

Aiming to unify known results about clustering mixtures of distributions under separation conditions, Kumar and Kannan [KK10] introduced a deterministic condition for clustering datasets. They showed that this single deterministic condition encompasses many previously studied clustering assumptions. More specifically, their proximity condition requires that in the target k-clustering, the projection of a point x onto the line joining its cluster center µ and some other center µ ′ , is a large additive factor closer to µ than to µ ′ . This additive factor can be roughly described as k times the spectral norm of the matrix representing the differences between the given (known) dataset and the means of the (unknown) target clustering. Clearly, the proximity condition implies center separation -the distance between any two centers must be as large as the above mentioned bound.In this paper we improve upon the work of Kumar and Kannan [KK10] along several axes. First, we weaken the center separation bound by a factor of √ k, and secondly we weaken the proximity condition by a factor of k (in other words, the revised separation condition is independent of k). Using these weaker bounds we still achieve the same guarantees when all points satisfy the proximity condition. Under the same weaker bounds, we achieve even better guarantees when only (1−ǫ)-fraction of the points satisfy the condition. Specifically, we correctly cluster all but a (ǫ + O(1/c 4 ))-fraction of the points, compared to O(k 2 ǫ)-fraction of [KK10], which is meaningful even in the particular setting when ǫ is a constant and k = ω(1). Most importantly, we greatly simplify the analysis of Kumar and Kannan. In fact, in the bulk of our analysis we ignore the proximity condition and use only center separation, along with the simple triangle and Markov inequalities. Yet these basic tools suffice to produce a clustering which (i) is correct on all but a constant fraction of the points, (ii) has k-means cost comparable to the k-means cost of the target clustering, and (iii) has centers very close to the target centers.Our improved separation condition allows us to match the results of the Planted Partition Model of McSherry [McS01], improve upon the results of Ostrovsky et al [ORSS06], and improve separation results for mixture of Gaussian models in a particular setting.

Impact of regularization on spectral clustering

“…The n × n edge probability matrix P = ((P ij )), given by (3), represents the population counterpart of the adjacency matrix A.…”

“…In our case, M = V τ . Recent results on cluster recovery using the K-means algorithm, as given in Kumar and Kannan [15] and Awasthi and Sheffet [3], provide conditions on X and M for the success of K-means. The following lemma is implied from Theorem 3.1 in Awasthi and Sheffet [3].…”

“…However, the results in [15], [3] can be extended to partitionŝ T that approximately minimize (5). The condition (7), called the center separation condition in [3], provides lower bounds on the pairwise distances between the population cluster centers that depend on the perturbation of data points around the population centers (represented by X − M ) and the cluster sizes.…”

Clustering under Perturbation Resilience

“…Here β is chosen to be 4 5 α since we are not able to prove the bound for β = α but will be able to when β is slightly smaller than α. Some other constant can be used instead of 4 5 . Definition 6.2.…”

“…Kumar and Kannan [19] proposed a proximity condition which assumes that in the target clustering, most data points satisfy that they are closer to their center than to any other center by an additive factor in the order of the maximal standard variance of their clusters in any direction. Their results are improved by Awasthi and Sheffet [4], which proposed a weaker version of the proximity condition called center separation, and designed algorithms achieving stronger guarantees under this weaker condition. These notions are not directly comparable to the perturbation resilience property.…”