Modified Fuzzy K-mean clustering using MapReduce in Hadoop and cloud

Garg, Dweepna; Gohil, Parth; Trivedi, Khushboo

doi:10.1109/icecct.2015.7226046

Cited by 7 publications

(4 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is supported by the fact that, during training there can be multiple chromosomes having similar fitness value that could force K-Means to undergo local minima and convergence where it can show updated centroid as any random outcome. Considering this fact, in this paper AGA-K-means estimates the number of chromosomes having similar fitness value which is further used to update and dynamically (8). (8) In Eq.…”

Section: Adaptive Ga Parameter Updatementioning

confidence: 99%

“…Considering this fact, in this paper AGA-K-means estimates the number of chromosomes having similar fitness value which is further used to update and dynamically (8). (8) In Eq. 8, the variables and signify the updated crossover and mutation probability, while its current probability is given by and .…”

Section: Adaptive Ga Parameter Updatementioning

confidence: 99%

“…Gheid [7] too recommended multiparty additive scheme for privacy preserved Big Data computation; however, this approach could be limited with heterogeneous datasets, which is common in Big Data or cloud applications. Author [5] found that parallel clustering concept can be better alternative for K-Means clustering, especially for Big Data analytics [3], [8]. In these approaches' authors introduced the concept of merging where data parallelization was performed by distributing data objects to all parallel processes.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Enhanced Evolutionary Computing Assisted K-Means Clustering Algorithm for BigData Analytics

Sarada¹

2020

IJATCSE

View full text Add to dashboard Cite

The exponential rise in internet technologies and allied applications has given rise to the technology named Big Data that intend to process gigantically huge data to assist real-time analytics or decision purposes. However, high pace increasing data heterogeneity, non-linearity, multi-dimensional features and unannotated data characteristics forces classical approaches to undergo huge computational overheads and limited accuracy that confines its suitability for major Big Data analytics purposes. With this motivation, in this paper a robust Big Data analytics model has been developed by incorporating Min-Max normalization, Dual Phased Feature Selection (DPFS) and enhanced Adaptive Genetic Algorithm (AGA) assisted K-Means clustering. Here, the use of Min-Max normalization helps alleviating key issues like data heterogeneity, data imbalance and pre-mature convergence during computation. Unlike classical feature selection approaches, DPFS exploited the efficacy of both Pearson correlation assisted significant test as well as T-Test analysis that ensure optimal feature selection for further computation. In addition, the use of AGA assisted K-Means clustering algorithm has accomplished computationally efficient and reliable clustering for efficient Big Data analytics purposes. Noticeably, the use of adaptive fitness sensitive GA parameter selection has strengthened our proposed system to exhibit better performance without imposing computational overheads. The computational efficacy of AGA-K-Means can strengthen MapReduce to be used for real-time Big Data analytics applications.

show abstract

Section: Adaptive Ga Parameter Updatementioning

confidence: 99%

Section: Adaptive Ga Parameter Updatementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Enhanced Evolutionary Computing Assisted K-Means Clustering Algorithm for BigData Analytics

Sarada¹

2020

IJATCSE

View full text Add to dashboard Cite

show abstract

“…Several other valuable research works and implementations over MapReduce can also be found in [16][17][18][19][20]. The use of sampling-based techniques as well as the possibility of applying one algorithm on a sample of the initial dataset and completing the clustering using another algorithm, usually form the basis for corresponding efficient implementations.…”

Section: Introductionmentioning

confidence: 99%

Efficient Big Text Data Clustering Algorithms using Hadoop and Spark

Gerakidis¹,

Megarchioti²,

Mamalis³

2021

IJCA

View full text Add to dashboard Cite

Document clustering is a traditional, efficient and yet quite effective, text mining technique when we need to get a better insight of the documents of a collection that could be grouped together. The K-Means algorithm and the Hierarchical Agglomerative Clustering (HAC) algorithm are two of the most known and commonly used clustering algorithms; the former due to its low time cost and the latter due to its accuracy. However, even the use of K-Means in text clustering over large-scale collections can lead to unacceptable time costs. In this paper we first address some of the most valuable approaches for document clustering over such 'big data' (large-scale) collections. We then present two very promising alternatives: (a) a variation of an existing K-Means-based fast clustering technique (known as BigKClustering -BKC) so that it can be applied in document clustering, and (b) a hybrid clustering approach based on a customized version of the Buckshot algorithm, which first applies a hierarchical clustering procedure on a sample of the input dataset and then it uses the results as the initial centers for a K-Means based assignment of the rest of the documents, with very few iterations. We also give highly efficient adaptations of the proposed techniques in the MapReduce model which are then experimentally tested using Apache Hadoop and Spark over a real cluster environment. As it comes out of the experiments, they both lead to acceptable clustering quality as well as to significant time improvements (compared to K-Means -especially the Buckshot-based algorithm), thus constituting very promising alternatives for big document collections.

show abstract