Relative clustering validity criteria: A comparative overview

Abstract

Many different relative clustering validity criteria exist that are very useful in practice as quantitative measures for evaluating the quality of data partitions, and new criteria have still been proposed from time to time. These criteria are endowed with particular features that may make each of them able to outperform others in specific classes of problems. In addition, they may have completely different computational requirements. Then, it is a hard task for the user to choose a specific criterion when he or she faces such a variety of possibilities. For this reason, a relevant issue within the field of clustering analysis consists of comparing the performances of existing validity criteria and, eventually, that of a new criterion to be proposed. In spite of this, the comparison paradigm traditionally adopted in the literature is subject to some conceptual limitations. The present paper describes an alternative, possibly complementary methodology for comparing clustering validity criteria and uses it to make an extensive comparison of the performances of 40 criteria over a collection of 962,928 partitions derived from five well-known clustering algorithms and 1080 different data sets of a given class of interest. A detailed review of the relative criteria under investigation is also provided that includes an original comparative asymptotic analysis of their computational complexities. This work is intended to be a complement of the classic study reported in 1985 by Milligan and Cooper as well as a thorough extension of a preliminary paper by the authors themselves. 

Clustering problems in a multiobjective framework

“…Measures to evaluate the results are: the Rand index [16], [18] and the Jaccard index [18]. These external indexes are, in general, used to measure similarity between two sets elements:…”

Fuzzy Clustering Algorithms and Validity Indices for Distributed Data

“…A common approach to quantitatively evaluate a data partition is based on the use of relative validity indices [51][52][53], which make it possible to compare different partitions in a relative manner [1]. Each candidate partition obtained by a clustering algorithm can be quantitatively evaluated by a relative index and compared to other partitions of the same data set [1,52,54], thereby also making it possible to estimate the number of clusters from data. This includes partitions produced by fuzzy clustering algorithms, which can be evaluated by means of fuzzy relative validity indices [2,54].…”

“…5.5.2) was chosen as I D to evaluate the quality of the partitions produced by the DOMR procedure. DFSS is the distributed fuzzy version of the Simplified Silhouette index [62], which scored the best among 40 indices in a recent comparative study [52,74]. The experiments were performed in a parallel scenario, where the responsibility for processing predetermined sub-collections of the data is distributed across the different cores of a single processor, which can then be seen as virtually distributed data sites, even though in this parallel computing setup the data is stored in a shared memory that is accessible by all the cores.…”

“…The experiments were run in a computer equipped with an AMD 3.6GHz processor with three processing modules, six cores and 16 GB-RAM using Linux 64 bits, Matlab 2009 and Matlab Parallel Computing Toolbox. Three data sets were generated for the experiments, using the same data set generator described in [52,75]. Each data set contains k D 8 clusters embedded in an n D 8 dimensional attribute space.…”

“…For example, the (dis)similarity measure used by an index may eventually need to be changed in order to work with different algorithms [61]. Another alternative is simply the use of the own objective function J of the algorithm (with its respective metric) as a validity index to find the best solution [70,71], e.g., by plotting a curve of such objective function values against the number of clusters and then looking for a knee in the chart [52].…”