A robust and efficient clustering algorithm based on cohesion self-merging

Abstract

Data clustering has attracted a lot of research attention in the field of computational statistics and data mining. In most related studies, the dissimilarity between two clusters is defined as the distance between their centroids, or the distance between two closest (or farthest) data points. However, all of these measurements are vulnerable to outliers, and removing the outliers precisely is yet another difficult task. In view of this, we propose a new similarity measurement, referred to as cohesion, to measure the inter-cluster distances. By using this new measurement of cohesion, we design a two-phase clustering algorithm, called cohesionbased self-merging (abbreviated as CSM), which runs in linear time to the size of input data set. Combining the features of partitional and hierarchical clustering methods, algorithm CSM partitions the input data set into several small subclusters in the first phase, and then continuously merges the subclusters based on cohesion in a hierarchical manner in the second phase. As shown by our performance studies, the cohesion-based clustering is very robust and possesses the excellent tolerance to outliers in various workloads. More importantly, algorithm CSM is shown to be able to cluster the data sets of arbitrary shapes very efficiently, and provide better clustering results than those by prior methods.

An incremental parallel neural network for unsupervised classification

“…If it does not exist (empty network) or if it does not satisfy the condition of equation 1, a new node is added with ) (t p w new . In the first layer, if the network is not empty the threshold is adjusted by the formula (2). Otherwise the threshold is infinite:…”

“…Figure 2. 2D noisy artificial data set used for the experiment As stated in [2] and [19], neither clustering algorithm can correctly partition such a data set nor eliminate noise from clusters.…”

“…A salient advantage of unsupervised learning is their ability to automatically partition a set of data into clusters [1] without any prior knowledge of classes. Most partition clustering algorithms run in linear time and work on large data sets [2]. The K-means algorithm is a local search procedure; it depends heavily on the initial starting conditions.…”

Adherence clustering: an efficient method for mining market-basket clusters

“…There are other fast algorithms designed for clustering large numerical data sets, such as CLARANS [24], BIRCH [25], DBSCAN [26], CURE [27], and CSM [28]. In addition, several approaches in [29][30][31] are proposed to solve the high dimensionality and data sparsity problems of numerical data.…”

On the Techniques for Data Clustering with Numerical Constraints

“…Data clustering is a useful technique for many applications, including similarity search, pattern recognition, trend analysis, marketing analysis, grouping, classification of documents, and so forth [3][7] [10]. In data clustering, similar data points are grouped together in a cluster.…”