Partitioning data into a finite number of k homogenous and separate clusters (groups) without use of prior knowledge is carried out by some unsupervised partitioning algorithm like the k-means clustering algorithm. To evaluate these resultant clusters for finding optimal number of clusters, properties such as cluster density, size, shape and separability are typically examined by some cluster validation methods. Mainly the aim of clustering analysis is to find the overall compactness of the clustering solution, for example variance within cluster should be a minimum and separation between the clusters should be a maximum. In this study, for k-means clustering we have developed a new method to find an optimal value of k number of clusters, using the features and variables inherited from datasets. The new proposed method is based on comparison of movement of objects forward/back from k to k+1 and k+1 to k set of clusters to find the joint probability, which is different from the other methods and indexes that are based on the distance. The performance of this method is also compared with some existing methods through two simulated datasets.

Abstract. Exploratory data analysis using data mining techniques is becoming more popular for investigating subtle relationships in health data, for which direct data collection trials would not be possible. Health data mining involving clustering for large complex data sets in such cases is often limited by insufficient key indicative variables. When a conventional clustering technique is then applied, the results may be too imprecise, or may be inappropriately clustered according to expectations. This paper suggests an approach which can offer greater range of choice for generating potential clusters of interest, from which a better outcome might in turn be obtained by aggregating the results. An example use case based on health services utilization characterization according to socio-demographic background is discussed and the blended clustering approach being taken for it is described.

No abstract