Finding Meaningful Cluster Structure Amidst Background Noise

Kushagra, Shrinu; Samadi, Samira; Ben-David, Shai

doi:10.1007/978-3-319-46379-7_23

Cited by 6 publications

(6 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Balcan et al (2008) introduce the list-decodable learning model, which was later studied by others, e.g. Balcan et al (2009) and Kushagra et al (2016). That work provides bounds for clustering in the presence of some adversarial data, but has two limitations relative to our results (apart from being in a somewhat different setting): the fraction of adversaries tolerated is small (O( 1 k )), and the bounds are not meaningful in high dimensions (e.g.…”

Section: Introductionmentioning

confidence: 96%

Learning from untrusted data

Charikar

Steinhardt

Valiant

2017

Proceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing

200

265

View full text Add to dashboard Cite

The vast majority of theoretical results in machine learning and statistics assume that the available training data is a reasonably reliable reflection of the phenomena to be learned or estimated. Similarly, the majority of machine learning and statistical techniques used in practice are brittle to the presence of large amounts of biased or malicious data. In this work we consider two frameworks in which to study estimation, learning, and optimization in the presence of significant fractions of arbitrary data.The first framework, list-decodable learning, asks whether it is possible to return a list of answers, with the guarantee that at least one of them is accurate. For example, given a dataset of n points for which an unknown subset of αn points are drawn from a distribution of interest, and no assumptions are made about the remaining (1 − α)n points, is it possible to return a list of poly(1/α) answers, one of which is correct? The second framework, which we term the semi-verified learning model, considers the extent to which a small dataset of trusted data (drawn from the distribution in question) can be leveraged to enable the accurate extraction of information from a much larger but untrusted dataset (of which only an α-fraction is drawn from the distribution).We show strong positive results in both settings, and provide an algorithm for robust learning in a very general stochastic optimization setting. This general result has immediate implications for robust estimation in a number of settings, including for robustly estimating the mean of distributions with bounded second moments, robustly learning mixtures of such distributions, and robustly finding planted partitions in random graphs in which significant portions of the graph have been perturbed by an adversary.

show abstract

Section: Introductionmentioning

confidence: 96%

Learning from untrusted data

Charikar

Steinhardt

Valiant

2017

Proceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing

200

265

View full text Add to dashboard Cite

show abstract

“…Kushagra et al [18] consider clusterability from the point of view of a structure in the data. They allow for noise in the data, but insist that the noise does not create structures by itself.…”

Section: The Problem Of Clusterability In the Previous Workmentioning

confidence: 99%

An Aposteriorical Clusterability Criterion for k-Means++ and Simplicity of Clustering

Kłopotek

2020

SN COMPUT. SCI.

View full text Add to dashboard Cite

We define the notion of a well-clusterable data set combining the point of view of the objective of k-means clustering algorithm (minimising the centric spread of data elements) and common sense (clusters shall be separated by gaps). We identify conditions under which the optimum of k-means objective coincides with a clustering under which the data is separated by predefined gaps.We investigate two cases: when the whole clusters are separated by some gap and when only the cores of the clusters meet some separation condition.We overcome a major obstacle in using clusterability criteria due to the fact that known approaches to clusterability checking had the disadvantage that they are related to the optimal clustering which is NP hard to identify.Compared to other approaches to clusterability, the novelty consists in the possibility of an a posteriori (after running k-means) check if the data set is well-clusterable or not. As the k-means algorithm applied for this purpose has polynomial complexity so does therefore the appropriate check. Additionally, if k-means++ fails to identify a clustering that meets clusterability criteria, with high probability the data is not well-clusterable.

show abstract

“…Kushagra, Samadi, and Ben-David [25] show that single linkage and a pruning technique are sufficient to find the target clustering under the condition that the data satisfies δ-center separation for δ ≥ 3.…”

Section: Definition 8 ([15]mentioning

confidence: 99%

Analysis of Ward's Method

Großwendt¹,

Röglin²,

Schmidt³

2019

Proceedings of the Thirtieth Annual ACM-SIAM Symposium on Discrete Algorithms

View full text Add to dashboard Cite

We study Ward's method for the hierarchical k-means problem. This popular greedy heuristic is based on the complete linkage paradigm: Starting with all data points as singleton clusters, it successively merges two clusters to form a clustering with one cluster less. The pair of clusters is chosen to (locally) minimize the k-means cost of the clustering in the next step.Complete linkage algorithms are very popular for hierarchical clustering problems, yet their theoretical properties have been studied relatively little. For the Euclidean k-center problem, Ackermann et al. [1] show that the k-clustering in the hierarchy computed by complete linkage has a worst-case approximation ratio of Θ(log k). If the data lies in R d for constant dimension d, the guarantee improves to O(1) [23], but the O-notation hides a linear dependence on d. Complete linkage for k-median or k-means has not been analyzed so far.In this paper, we show that Ward's method computes a 2-approximation with respect to the k-means objective function if the optimal k-clustering is well separated. If additionally the optimal clustering also satisfies a balance condition, then Ward's method fully recovers the optimum solution. These results hold in arbitrary dimension. We accompany our positive results with a lower bound of Ω((3/2) d ) for data sets in R d that holds if no separation is guaranteed, and with lower bounds when the guaranteed separation is not sufficiently strong. Finally, we show that Ward produces an O(1)approximative clustering for one-dimensional data sets.

show abstract

Finding Meaningful Cluster Structure Amidst Background Noise

Cited by 6 publications

References 10 publications

Learning from untrusted data

Learning from untrusted data

An Aposteriorical Clusterability Criterion for k-Means++ and Simplicity of Clustering

Analysis of Ward's Method

Contact Info

Product

Resources

About