A Self-Adaptive Fuzzyc-Means Algorithm for Determining the Optimal Number of Clusters

Abstract

For the shortcoming of fuzzy c-means algorithm (FCM) needing to know the number of clusters in advance, this paper proposed a new self-adaptive method to determine the optimal number of clusters. Firstly, a density-based algorithm was put forward. The algorithm, according to the characteristics of the dataset, automatically determined the possible maximum number of clusters instead of using the empirical rule n and obtained the optimal initial cluster centroids, improving the limitation of FCM that randomly selected cluster centroids lead the convergence result to the local minimum. Secondly, this paper, by introducing a penalty function, proposed a new fuzzy clustering validity index based on fuzzy compactness and separation, which ensured that when the number of clusters verged on that of objects in the dataset, the value of clustering validity index did not monotonically decrease and was close to zero, so that the optimal number of clusters lost robustness and decision function. Then, based on these studies, a self-adaptive FCM algorithm was put forward to estimate the optimal number of clusters by the iterative trial-and-error process. At last, experiments were done on the UCI, KDD Cup 1999, and synthetic datasets, which showed that the method not only effectively determined the optimal number of clusters, but also reduced the iteration of FCM with the stable clustering result.

Automated detection of photoreceptor disruption in mild diabetic retinopathy on volumetric optical coherence tomography

“…If clusters are too few, the algorithm would assign an artificially large area to the EZ defect, and vice versa, if clusters are too many, some pixels in the EZ defect area would be lost. Numerous methods have been proposed to solve this issue, such as evaluation of clustering quality by a validity index [47] based on either the degree of separation [48,49] or the ratio between separation and compactness [50]; finding density peaks [51] and hybrid solutions [52]; but ultimately the optimal clustering method is highly dependent on the application. In our algorithm, we have implemented a novel clustering method that optimizes a compound image formed by the retina under scrutiny stitched to a reference from healthy retinas with characteristics known a priori, attaining a high sensitivity and specificity of EZ defect area.…”

A Fast Method for Estimating the Number of Clusters Based on Score and the Minimum Distance of the Center Point

“…Min Ren et al found that there was a problem with the improved index by Bensaid et al When c → n, the index will decrease monotonously, approaching 0, lacking the robustness to determine the optimal number of clusters. Therefore, Min Ren et al proposed an improved Xie-Beni index [2] (Xie-Beni index with punitive measures) in 2016 to determine the optimal cluster number. Suneel proposed in 2018 to use diversity to determine the number of clusters [13], in other words, to encourage balanced clustering through cluster size and the distance between clusters.…”

“…Clustering analysis has a long history and is a key technology of data analysis. As an unsupervised learning algorithm [1], it does not require prior knowledge and has been widely applied to image processing, business intelligence, network mining, and other fields [2,3]. A clustering algorithm is used to divide data into multiple clusters and make the elements within clusters as similar as possible and the elements between clusters as different as possible.…”

Fuzzy Clustering with Spatial Correction and Its Application to Geometallurgical Domaining

“…There are two hyper-parameters that need to be defined to apply SWFC: the fuzzier m and the parameter λ for weights. Most cases reported in the literature suggest that a value of 2.0 for m is a reasonable choice to account for uncertainty (Pal and Bezdek 1995;Ren et al 2016), and m = 2.0 is used in all applications of SWFC discussed in this paper. For parameter λ, there is no rule of thumb guidance in the literature.…”