MapReduce-enhanced Fuzzy K-Least Medians for Qualitative Clustering of Document Big Data

Abstract

Researchers design MapReduce-enhanced versions of traditional clustering algorithms to obtain the time performance benefit in big data clustering. However, the current literature shows that only a handful of partitioning clustering algorithms are enhanced using the MapReduce model. This work proposes a MapReduce-enhanced design of the Fuzzy K-Least Medians algorithm (MRFK-LstMdns) and its two novel variations. The purpose is to determine the best-performing MapReduce-enhanced partitioning clustering algorithms, among the proposed and existing ones, in terms of time performance and cluster quality. The work first preprocesses different sized self crawled document datasets. Then an optimal noise removal process is employed to make the dataset optimally noise free. The proposed MRFK-LstMdns and its two novel variations are designed using three MapReduce job chaining. Each of the jobs performs the staged and suitable algorithmic parts. The time performance and cluster quality of the proposed algorithms are compared against the existing MapReduce-enhanced partitioning algorithms. Although the proposed algorithm's time complexity is higher than the existing algorithms, the suitable chained MapReduce job design of the proposed algorithms makes them consume existing algorithms like execution time. A majority voting technique using seven cluster quality measures shows that the MRFK-LstMdns generates better quality clusters than existing algorithms.