A geometric approach to cluster validity for normal mixtures

Abstract

We study indices for choosing the correct number of components in a mixture of normal distributions. Previous studies have been confined to indices based wholly on probabilistic models. Viewing mixture decomposition as probabilistic clustering (where the emphasis is on partitioning for geometric substructure) as opposed to parametric estimation enables us to introduce both fuzzy and crisp measures of cluster validity for this problem. We presume the underlying samples to be unlabeled, and use the expectation-maximization (EM) algorithm to find clusters in the data. We test 16 probabilistic, 3 fuzzy and 4 crisp indices on 12 data sets that are samples from bivariate normal mixtures having either 3 or 6 components. Over three run averages based on different initializations of EM, 10 of the 23 indices tested for choosing the right number of mixture components were correct in at least 9 of the 12 trials. Among these were the fuzzy index of Xie-Beni, the crisp Davies-Bouldin index, and two crisp indices that are recent generalizations of Dunn's index.

Finding Alternative Clusterings Using Constraints

“…This is one experimental setup used in [1] and has the benefit of producing an algorithm independent initial clustering which the algorithms will attempt to find a good and alternative clustering to. The DI is a measure of the minimum distance between two clusters calculated as the average distance between each pair of points that are in different clusters, this is then normalized by the maximum cluster diameter [4]. We report the VQE as it is the objective function that k-means minimizes and allows us to compare our results with the work described in [5].…”

Self-organization of interorganizational process design

“…To compare the quality of the different cluster analyses the Dunn index has been found a useful instrument (Bezdek et al 1997). It is the ratio of the minimum inter-cluster distance to the maximum cluster diameter.…”

Fuzzy Cluster Validation Using the Partition Negentropy Criterion

“…In principle, they could also be used as validity indices in model clustering, to compare models with different number of components and thus automatically select the number of clusters. This possibility has been deeply explored in the literature, and different indices based on these and similar criteria have been proposed to validate clustering partitions and assess the number of components in clustering problems using mixture models [7,8,20]. In particular, the Information Completed Likelihood (ICL) [21,22], which is essentially the BIC criterion penalized by subtraction of the estimated partition mean entropy, has been shown to outperform AIC and BIC when the focus is clustering rather than density estimation.…”