Robust Clustering Using Outlier-Sparsity Regularization

Abstract

Notwithstanding the popularity of conventional clustering algorithms such as K-means and probabilistic clustering, their clustering results are sensitive to the presence of outliers in the data. Even a few outliers can compromise the ability of these algorithms to identify meaningful hidden structures rendering their outcome unreliable. This paper develops robust clustering algorithms that not only aim to cluster the data, but also to identify the outliers. The novel approaches rely on the infrequent presence of outliers in the data which translates to sparsity in a judiciously chosen domain. Capitalizing on the sparsity in the outlier domain, outlier-aware robust K-means and probabilistic clustering approaches are proposed. Their novelty lies on identifying outliers while effecting sparsity in the outlier domain through carefully chosen regularization. A block coordinate descent approach is developed to obtain iterative algorithms with convergence guarantees and small excess computational complexity with respect to their non-robust counterparts. Kernelized versions of the robust clustering algorithms are also developed to efficiently handle high-dimensional data, identify nonlinearly separable clusters, or even cluster objects that are not represented by vectors. Numerical tests on both synthetic and real datasets validate the performance and applicability of the novel algorithms.Comment: Submitted to IEEE Trans. on PAM

Vector quantization based on ε-insensitive mixture models

“…For each s, we applied the EIMMs with K ¼ 2; 4; …; 48; 50 and adopted the number of components K when the increase in the likelihood was saturated. The resulting rate (10) and average distortion (11) were calculated for the two data sets, the Laplacian data set ( Fig. 1(a)) and the Gaussian data set ( Fig.…”

“…In particular, Laplacian mixture models (LMMs) have been proposed and applied for the purposes of robust clustering and overcomplete source separation [6,14]. Among robust clustering methods [11,10], those based on LMMs provide simple learning algorithms similar to the learning of Gaussian mixture models (GMMs). However, there are two drawbacks in LMMs: (1) the degree of the robustness is uncontrollable, and (2) a cluster mean vector can inappropriately converge to a data sample, which is caused by the nature of the absolute-loss function.…”

Robust clustering of data collected via crowdsourcing

“…As a result, the proposed algorithm will allow for (a) soft assignments of data points to clusters; (b) rating of annotators; and, (c) estimating the number of Gaussian components in the mixture model based on the algorithm developed in [7] for a Gaussian mixture only. Relative to prior works in robust clustering [8][9][10][11][12], the present contribution accounts for the variable reliability of data to be clustered, which is a distinct feature of crowdsourcing.…”

Sparse adaptive possibilistic clustering

“…In [11] the clusters are recovered in a sequential manner, in contrast to [10] (and all previous algorithms), where clusters are recovered simultaneously. Other such methods are given in [12], [13].…”