Large-Scale Automatic K-Means Clustering for Heterogeneous Many-Core Supercomputer

Abstract

This paper presents an automatic k-means clustering solution targeting the Sunway TaihuLight supercomputer. We first introduce a multi-level parallel partition approach that not only partitions by dataflow and centroid, but also by dimension, which unlocks the potential of the hierarchical parallelism in the heterogeneous many-core processor and the system architecture of the supercomputer. The parallel design is able to process large-scale clustering problems with up to 196,608 dimensions and over 160,000 targeting centroids, while maintaining high performance and high scalability. Furthermore, we propose an automatic hyper-parameter determination process for k-means clustering, by automatically generating and executing the clustering tasks with a set of candidate hyper-parameter, and then determining the optimal hyper-parameter using a proposed evaluation method. The proposed auto-clustering solution can not only achieve high performance and scalability for problems with massive high-dimensional data, but also support clustering without sufficient prior knowledge for the number of targeted clusters, which can potentially increase the scope of k-means algorithm to new application areas.

Parallel and accurate k‐means algorithm on CPU‐GPU architectures for spectral clustering

“…Although the KMeans API is supposed to be highly optimized and fast, it turns out that our GPU code appears to faster than the KMeans of RAPIDS v0.19 on RTX 2080Ti. We guess this significant difference is mainly induced by the wrapper overhead of Python interface (as our tests do not last long) and by the youth of RAPIDS (v0.19 in April 2021). k ‐Means of Yu et al 16 on one node of Sunway TaihuLight supercomputer . The SW26010 manycore processor offers 260 cores and has a significantly different design from other multicore and manycore processors 16 .…”

“…We guess this significant difference is mainly induced by the wrapper overhead of Python interface (as our tests do not last long) and by the youth of RAPIDS (v0.19 in April 2021). k ‐Means of Yu et al 16 on one node of Sunway TaihuLight supercomputer . The SW26010 manycore processor offers 260 cores and has a significantly different design from other multicore and manycore processors 16 . This processor appeared in 2016 as part of the Sunway TaihuLight supercomputer which was at that time ranked #1 in the Top500 list from 2016 to 2018.…”

“…Note that the -Ofast flag introduces strong optimizations in floating-point computations (like the -ffast-math flag) but they are supported by our code. Then the nearest centroid for each instance can be found and recorded (lines [16][17][18]. Finally the cluster label of each instance is updated according to its nearest centroid, and the changes of label are counted into the private track of each thread (lines [22][23][24][25].…”

“…We also address the numerical accuracy issue when processing large datasets in single precision. To our knowledge (e.g., in References 12‐16), there is no specific study about the numerical accuracy issue of the update phase due to the propagation of round‐off errors. However, we observed this issue when processing large datasets with floating‐point numbers in single precision arithmetic, that may lead to poor clustering quality.…”

Quality evaluation system of engineering cost education curriculum based on data clustering

“…The current teaching quality evaluation mostly adopts the method of student evaluation. Usually, the educational administration department publishes the teacher's teaching quality evaluation form on the Internet at the mid-term or end of the period, scores the teacher according to the evaluation items in the evaluation form, and determines the teacher's teaching quality evaluation grade according to the evaluation results after statistics by the educational administration department [5]. This evaluation method can only obtain simple evaluation results, can not analyze the evaluation data, can not give full play to the guiding role of teaching evaluation in teaching, and does not make full use of the existing data.…”