Clustering for metric and nonmetric distance measures

Abstract

We study a generalization of the k -median problem with respect to an arbitrary dissimilarity measure D. Given a finite set P of size n , our goal is to find a set C of size k such that the sum of errors D( P,C ) = ∑ p ∈ P min c ∈ C {D( p,c )} is minimized. The main result in this article can be stated as follows: There exists a (1+ϵ)-approximation algorithm for the k -median problem with respect to D, if the 1-median problem can be approximated within a factor of (1+ϵ) by taking a random sample of constant size and solving the 1-median problem on the sample exactly. This algorithm requires time n 2 O ( mk log( mk /ϵ)), where m is a constant that depends only on ϵ and D. Using this characterization, we obtain the first linear time (1+ϵ)-approximation algorithms for the k -median problem in an arbitrary metric space with bounded doubling dimension, for the Kullback-Leibler divergence (relative entropy), for the Itakura-Saito divergence, for Mahalanobis distances, and for some special cases of Bregman divergences. Moreover, we obtain previously known results for the Euclidean k -median problem and the Euclidean k -means problem in a simplified manner. Our results are based on a new analysis of an algorithm of Kumar et al. [2004].

Informative Energy Metric for Similarity Measure in Reproducing Kernel Hilbert Spaces

“…However, the non-metric distances will lead to inconsistency and conflict, when the metric violate one or more metric axioms as in Refs. 3,4 . So it is necessary to change this metric distance.…”

“…In these applications, a notion of similarity is induced by computing kernel functions on arbitrary training sample pairs in input space. However, many similarity measures are developed based on metric distance under highdimensional setting and samples in the RKHS often violate one or more metric axioms 3,4 , this may impair the performance of machine learning algorithms. Therefore, how to choose a "good" similarity measure is one of the key concerns of these algorithms.…”

“…Although they have been widely used in many applications, they may violate one or more metric axioms under high-dimensional setting, which is called non-metric distance 3,4,7,8 . As indicated by Cover and Thomas in Ref.…”

Mixed Bregman Clustering with Approximation Guarantees

“…Previously, approximation bounds for Bregman clustering algorithms have been given by [CM08] and [ABS08]. Chadhuri and McGregor [CM08] consider the KL divergence, which is a particularly interesting case as the KL divergence between two members of the same exponential family is a Bregman divergence between their natural parameters [BMDG05].…”

“…Chadhuri and McGregor [CM08] consider the KL divergence, which is a particularly interesting case as the KL divergence between two members of the same exponential family is a Bregman divergence between their natural parameters [BMDG05]. Ackermann et al [ABS08] consider a statistically defined class of distortion measures which includes the KL divergence and other Bregman divergences. In both of these cases, the algorithms achieve (1 + ε)-approximation for arbitrary ε > 0.…”