Utilizing the buckshot algorithm for efficient big data clustering in the MapReduce model

Gerakidis, Sergios; Mamalis, Basilis

doi:10.1145/3368640.3368658

Cited by 1 publication

(1 citation statement)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…More specifically, we initially follow the idea of the Buckshot clustering algorithm given in [16], which first executes a hierarchical algorithm on a sample of the input set of data and then applies the results as initial centroids for a KMeans based routine with very small number of iterations. Furthermore, we develop an adapted version of the Buckshot approach suitable for (big) text data, and we finally present a highly efficient implementation of the proposed Buckshot-based adaptation in the MapReduce model (see also [31]).…”

Section: Introductionmentioning

confidence: 99%

Efficient Big Text Data Clustering Algorithms using Hadoop and Spark

Gerakidis¹,

Megarchioti²,

Mamalis³

2021

IJCA

Self Cite

View full text Add to dashboard Cite

Document clustering is a traditional, efficient and yet quite effective, text mining technique when we need to get a better insight of the documents of a collection that could be grouped together. The K-Means algorithm and the Hierarchical Agglomerative Clustering (HAC) algorithm are two of the most known and commonly used clustering algorithms; the former due to its low time cost and the latter due to its accuracy. However, even the use of K-Means in text clustering over large-scale collections can lead to unacceptable time costs. In this paper we first address some of the most valuable approaches for document clustering over such 'big data' (large-scale) collections. We then present two very promising alternatives: (a) a variation of an existing K-Means-based fast clustering technique (known as BigKClustering -BKC) so that it can be applied in document clustering, and (b) a hybrid clustering approach based on a customized version of the Buckshot algorithm, which first applies a hierarchical clustering procedure on a sample of the input dataset and then it uses the results as the initial centers for a K-Means based assignment of the rest of the documents, with very few iterations. We also give highly efficient adaptations of the proposed techniques in the MapReduce model which are then experimentally tested using Apache Hadoop and Spark over a real cluster environment. As it comes out of the experiments, they both lead to acceptable clustering quality as well as to significant time improvements (compared to K-Means -especially the Buckshot-based algorithm), thus constituting very promising alternatives for big document collections.

show abstract

Section: Introductionmentioning

confidence: 99%