Compressive Clustering of High-Dimensional Data

Abstract

In this paper we focus on realistic clustering problems where the input data is high-dimensional and the clusters have complex, multimodal distribution. In this challenging setting the conventional methods, such as k-centers family, hierarchical clustering or those based on model fitting, are inefficient and typically converge far from the globally optimal solution. As an alternative, we propose a novel unsupervised learning approach which is based on the compressive sensing paradigm. The key idea underlying our algorithm is to monitor the distance between the test sample and its principal projection in each cluster, and continue re-assigning it to the cluster yielding the smallest residual. As a result, we obtain an iterative procedure which, under the compressive assumptions, minimizes the total reconstruction error of all samples from their nearest clusters. To evaluate the proposed approach, we have conducted a series of experiments involving various image collections where the task was to automatically group similar objects. Comparison of the obtained results with those yielded by the state-of-the-art clustering methods provides evidence for high discriminative power of our algorithm. International Conference on Machine Learning and Applications (ICMLA)This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. Abstract-In this paper we focus on realistic clustering problems where the input data is high-dimensional and the clusters have complex, multimodal distribution. In this challenging setting the conventional methods, such as k-centers family, hierarchical clustering or those based on model fitting, are inefficient and typically converge far from the globally optimal solution. As an alternative, we propose a novel unsupervised learning approach which is based on the compressive sensing paradigm. The key idea underlying our algorithm is to monitor the distance between the test sample and its principal projection in each cluster, and continue re-assigning it to the cluster yielding the smallest residual. As a result, we obtain an iterative procedure which, under the compressive assumptions, minimizes the total reconstruction error of all samples from their nearest clusters. To evaluate the proposed approach, we have conducted a series of experiments involving various image collections where the task was to automatically group similar objects. Comparison of the obtained results with thos...

Advanced K-means clustering algorithm for large ECG data sets based on K-SVD approach

“…Keeping the first parameter's value fixed, we minimize the second parameter; and then do vice versa. Thus the new of c is found as [23]: c= ∑ i ∑ j ǁX j -v j (X i )ǁ 2 (16) After applying (16) to all datasets, we are able to find out minimum value in (14). Following is the pseudo-code of the proposed algorithm.…”

“…In the K-SVD algorithm, we solve (8) iteratively, using two stages, parallel to those in K-Means [22]. The initial step of the dictionary training method can be selected by taking ( dimensional) K N vectors of large ECG dataset as Y by generating K atoms randomly [23,24]. The stop condition can be defined either as the time of iteration process or error ratio or the combination of both.…”

“…Therefore, the goal of K-Mean clustering algorithm based on K-SVD approach is to find an optimal set of sparsity bases for each data point such that the sum of their reconstruction errors is minimized [23]. For a given vector i X and a cluster membership function, the best cluster assignment can be determined by checking how well this vector gets reconstructed based on the found sparse vector S with set to zero all coefficients not corresponding to the basis vectors belonging to each th j cluster.…”

“…For a given vector i X and a cluster membership function, the best cluster assignment can be determined by checking how well this vector gets reconstructed based on the found sparse vector S with set to zero all coefficients not corresponding to the basis vectors belonging to each th j cluster. From this way K cluster are reconstructed as [23]: ( ) ,…”

Fast clustering algorithm for large ECG data sets based on CS theory in combination with PCA and K-NN methods

“…The reconstruction process of feature vector is done by solving a 1 minimization  least-square problem as [19]:…”