An Improved Hierarchical Clustering Algorithm Based on the Idea of Population Reproduction and Fusion

Abstract

Aiming to resolve the problems of the traditional hierarchical clustering algorithm that cannot find clusters with uneven density, requires a large amount of calculation, and has low efficiency, this paper proposes an improved hierarchical clustering algorithm (referred to as PRI-MFC) based on the idea of population reproduction and fusion. It is divided into two stages: fuzzy pre-clustering and Jaccard fusion clustering. In the fuzzy pre-clustering stage, it determines the center point, uses the product of the neighborhood radius eps and the dispersion degree fog as the benchmark to divide the data, uses the Euclidean distance to determine the similarity of the two data points, and uses the membership grade to record the information of the common points in each cluster. In the Jaccard fusion clustering stage, the clusters with common points are the clusters to be fused, and the clusters whose Jaccard similarity coefficient between the clusters to be fused is greater than the fusion parameter jac are fused. The common points of the clusters whose Jaccard similarity coefficient between clusters is less than the fusion parameter jac are divided into the cluster with the largest membership grade. A variety of experiments are designed from multiple perspectives on artificial datasets and real datasets to demonstrate the superiority of the PRI-MFC algorithm in terms of clustering effect, clustering quality, and time consumption. Experiments are carried out on Chinese household financial survey data, and the clustering results that conform to the actual situation of Chinese households are obtained, which shows the practicability of this algorithm.

Clustering Explanation based on Multi-Hyperrectangle

“…The specific calculation measures the correlation between the two by calculating and standardizing their mutual information. The value of NMI ranges from 0 to 1, and the higher the value, the more consistent the clustering results are with the real labels 35 . The formula for calculating NMI is more complicated and usually uses matrix operation.…”

Advanced Machine Learning Applications in Big Data Analytics

“…Yin et al [15] proposed an improved hierarchical clustering algorithm called PRI-MFC to solve the problems of traditional hierarchical clustering algorithms. The algorithm was tested on artificial and real datasets and the experimental results showed superiority in clustering effect, quality, and time consumption.…”

Spectral Clustering Approach with K-Nearest Neighbor and Weighted Mahalanobis Distance for Data Mining

“…When the consistency graph converges, the spectral clustering algorithm is performed on it to obtain the final clustering effect. Yin et al [34] determined the center point in the fuzzy pre-clustering stage, used the product of the neighborhood radius eps and the dispersion degree fog as a benchmark to divide the data, used the Euclidean distance to determine the similarity between two data points, and used the degree of membership to record the information of common points in each cluster. In addition, some new methods have also been proposed in recent years [35][36][37][38][39][40][41][42][43][44].…”