A new coreset framework for clustering

Abstract

In all state-of-the-art sketching and coreset techniques for clustering, as well as in the best known fixed-parameter tractable approximation algorithms, randomness plays a key role. For the classic k-median and k-means problems, there are no known deterministic dimensionality reduction procedure or coreset construction that avoid an exponential dependency on the input dimension d, the precision parameter ε −1 or k. Furthermore, there is no coreset construction that succeeds with probability 1 − 1/n and whose size does not depend on the number of input points, n. This has led researchers in the area to ask what is the power of randomness for clustering sketches [Feldman WIREs Data Mining Knowl. Discov'20].Similarly, the best approximation ratio achievable deterministically without a complexity exponential in the dimension are 1 + √ 2 for k-median [Cohen-Addad, Esfandiari, Mirrokni, Narayanan, STOC'22] and 6.12903 for k-means [Grandoni, Ostrovsky, Rabani, Schulman, Venkat, Inf. Process. Lett. '22]. Those are the best results, even when allowing a complexity FPT in the number of clusters k: this stands in sharp contrast with the (1 + ε)-approximation achievable in that case, when allowing randomization.In this paper, we provide deterministic sketches constructions for clustering, whose size bounds are close to the best-known randomized ones. We show how to compute a dimension reduction onto ε −O(1) log k dimensions in time k O(ε −O(1) +log log k) poly(nd), and how to build a coreset of size O k 2 log 3 kε −O(1) in time 2 ε O(1) k log 3 k + k O(ε −O(1) +log log k) poly(nd). In the case where k is small, this answers an open question of [Feldman WIDM'20] and [Munteanu and Schwiegelshohn, Künstliche Intell. '18] on whether it is possible to efficiently compute coresets deterministically.We also construct a deterministic algorithm for computing (1+ε)-approximation to k-median and k-means in high dimensional Euclidean spaces in time 2 k 2 log 3 k/ε O(1) poly(nd), close to the best randomized complexity of 2 (k/ε) O(1) nd (see [Kumar, Sabharwal, Sen, JACM 10] and [Bhattacharya, Jaiswal, Kumar, TCS'18]).Furthermore, our new insights on sketches also yield a randomized coreset construction that uses uniform sampling, that immediately improves over the recent results of [Braverman et al. FOCS '22] by a factor k.

Towards Optimal Lower Bounds for k-median and k-means Coresets

“…Our result improves over the Ω(kε −1 log n) lower bound of Baker, Braverman, Huang, Jiang, Krauthgamer, and Wu [6]. For the k-median and k-means objective matches the upper bounds proposed in Feldman and Langberg [41] and Cohen-Addad, Saulpic, and Schwiegelshohn [35] up to polylog(1/ε) factors.…”

“…For any ε, k, D such that D ≥ 5 log 1/ε and log k = O(D), there exists a graph with doubling dimension D on which any (ε, k, z)-coreset using offset ∆ must have size Ω kD ε 2 . This matches up to polylog(1/ε) factors the upper bound from [35] for k-median and k-means.…”

“…Our result Discrete Metrics O(kε − max (2,z) log n) [35] Ω(kε −1 log n) [6] Ω(kε −2 log n)* with doubling dimension D O(kε − max(2,z) D) [35] -Ω(kε −2 D)*…”

“…Euclidean O(kε −2−max (2,z) ) [35] O(k 2 ε −4 ) [17] Ω(k2 z/100 ) [55] O z (kε −2 • min(ε −z , k)) Ω(kε −2 )…”

Coresets for Clustering with Missing Values

“…In Euclidean space, recent work showed that coresets for k-MEANS and k-MEDIAN clustering can have size that is independent of the Euclidean dimension [FSS20, SW18, HV20]. Beyond Euclidean space, coresets of size independent of the data-set size were constructed also for many important metric spaces [HJLW18,BJKW21,CASS21]. A more comprehensive overview can be found in recent surveys [Phi17,Fel20].…”