An empirical comparison between stochastic and deterministic centroid initialisation for K-means variations

Abstract

K-Means is one of the most used algorithms for data clustering and the usual clustering method for benchmarking. Despite its wide application it is well-known that it suffers from a series of disadvantages; it is only able to find local minima and the positions of the initial clustering centres (centroids) can greatly affect the clustering solution. Over the years many K-Means variations and initialisation techniques have been proposed with different degrees of complexity. In this study we focus on common K-Means variations along with a range of deterministic and stochastic initialisation techniques. We show that, on average, more sophisticated initialisation techniques alleviate the need for complex clustering methods. Furthermore, deterministic methods perform better than stochastic methods. However, there is a trade-off: less sophisticated stochastic methods, executed multiple times, can result in better clustering. Factoring in execution time, deterministic methods can be competitive and result in a good clustering solution. These conclusions are obtained through extensive benchmarking using a range of synthetic model generators and real-world data sets.

Strategies discovery in the active allothetic place avoidance task

“…We applied K-Means clustering on these transformed data and the original dataset, varying the number of clusters from 1 to 10. As a clustering algorithm, we used Lloyd’s K-Means 37 initialised with the Density K-Means++ method 38 which worked well in our previous benchmark 39 . We used the R package clustree 40 to visualize a K-Means clustering tree (number of target clusters 1 to 10) for data transformations with different numbers of PCs (2 to 11, leading to different data dimensionality, see also Supplementary Material ).…”

An Effective Initialization Method Based on Quartiles for the K-means Algorithm

“…SSE, Silhouette score, Purity and CPU time are used to measure the performance of our proposed method. While SSE, Purity and Silhouette score measure the quality of the clusters formed, CPU Time measures efficiency (7,(13)(14)(15) . These evaluation criteria are explained below:…”

“…2. Purity: It is an external validity index that measures the degree of similarity between the clustering solution formed by a clustering method and that specified by the given class labels (7).…”

“…Various deterministic and non-deterministic initialization methods have been analyzed and their characteristics explained in literature (5,7) . While, non-deterministic initialization methods often produce highly variable results across multiple runs and need to be executed multiple times and the output of the run that gives the least SSE is taken as the result; deterministic methods produce identical results across multiple runs and need to be executed only once.…”

“…However, may not be helpful in case of the fuzzy c-means and k-means frameworks as these employ the mean as their center of gravity. In (7) , an empirical comparison is carried out among the various stochastic and deterministic methods of centroid initialization including random and K-means++, for the K-means variations on synthetic data. They demonstrate that, on average, more advanced initialization strategies reduce the requirement for intricate clustering strategies.…”