How Fast Is the k-Means Method?

Abstract

We present polynomial upper and lower bounds on the number of iterations performed by the k-means method (a.k.a. Lloyd's method) for k-means clustering. Our upper bounds are polynomial in the number of points, number of clusters, and the spread of the point set. We also present a lower bound, showing that in the worst case the k-means heuristic needs to perform Ω(n) iterations, for n points on the real line and two centers. Surprisingly, the spread of the point set in this construction is polynomial. This is the first construction showing that the k-means heuristic requires more than a polylogarithmic number of iterations. Furthermore, we present two alternative algorithms, with guaranteed performance, which are simple variants of the k-means method. Results of our experimental studies on these algorithms are also presented.

Anomaly Characterization in Large Scale Networks

“…All the nodes of the system are continuously organized into clusters computed through the k-means algorithm exclusively run by the management node, which is a clear impediment to the scalability of their approach. Other works aim at minimizing the processing cost for continuous monitoring [13], [9], [14] in the light of the theoretical results of [5], however similarly to [15], all these approaches suffer from a centralized handling of the clustering process. Recently, Choffnes et al [2] have proposed to leverage structured peer-to-peer architectures (i.e., Distributed Hashing Tables) to guarantee efficient and scalable monitoring management.…”

The Planar k-Means Problem is NP-Hard

“…Har-Peled and Sadri [11] and Arthur and Vassilvitskii [5,4] examine the question of how quickly this algorithm and its variants converge to a local optimum. Lloyd's algorithm also does not provide any significant guarantee about how well the solution that it computes approximates the optimal solution.…”

k-means Requires Exponentially Many Iterations Even in the Plane

“…Other bounds are known for the special case d = 1. Namely, for the one-dimensional case, Har-Peled and Sadri [9] provided a worst-case lower bound of Ω(n), and showed an upper bound of O(nΔ 2 ), where Δ is the spread of the point set (i.e., the ratio between the largest and the smallest pairwise distance). They also conjectured that k-means might run in time polynomial in n and Δ for any d.…”

“…Arthur and Vassilvitskii [2] showed that k-means can run for super-polynomially many iterations, improving the best known lower bound from Ω(n) [10] Also they show that their construction can be modified to have low spread, disproving the aforementioned conjecture in [9] for d = Ω( √ n). A more recent line of work that aims to close the gap between practical and theoretical performance makes use of the smoothed analysis introduced by Spielman and Teng [15].…”