A Comprehensive Study of Clustering Algorithms for Big Data Mining with MapReduce Capability

Pandey, Kamlesh Kumar; Shukla, Diwakar; Milan, Ram

doi:10.1007/978-981-15-2071-6_34

Cited by 7 publications

(7 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some authors (Fahad et al. , 2014; Pandey et al. , 2020) defined the clustering algorithm characteristics for big data using volume, variety and velocity characteristics.…”

Section: Related Work Of Initialization Methodsmentioning

confidence: 99%

“…Some authors (Fahad et al, 2014;Pandey et al, 2020) defined the clustering algorithm characteristics for big data using volume, variety and velocity characteristics. The volume Clustering for big data mining characteristic defines the scale, outliers and dimensionality of the dataset; variety specifies the data forms, cluster structure and shape; and velocity represents the computing efficiency through time complexity and scalability of the clustering algorithm.…”

Section: Analysis Of Existing Initialization Algorithm For Big Data C...mentioning

confidence: 99%

See 1 more Smart Citation

NDPD: an improved initial centroid method of partitional clustering for big data mining

Pandey

Shukla

2022

JAMR

Self Cite

View full text Add to dashboard Cite

PurposeThe K-means (KM) clustering algorithm is extremely responsive to the selection of initial centroids since the initial centroid of clusters determines computational effectiveness, efficiency and local optima issues. Numerous initialization strategies are to overcome these problems through the random and deterministic selection of initial centroids. The random initialization strategy suffers from local optimization issues with the worst clustering performance, while the deterministic initialization strategy achieves high computational cost. Big data clustering aims to reduce computation costs and improve cluster efficiency. The objective of this study is to achieve a better initial centroid for big data clustering on business management data without using random and deterministic initialization that avoids local optima and improves clustering efficiency with effectiveness in terms of cluster quality, computation cost, data comparisons and iterations on a single machine.Design/methodology/approachThis study presents the Normal Distribution Probability Density (NDPD) algorithm for big data clustering on a single machine to solve business management-related clustering issues. The NDPDKM algorithm resolves the KM clustering problem by probability density of each data point. The NDPDKM algorithm first identifies the most probable density data points by using the mean and standard deviation of the datasets through normal probability density. Thereafter, the NDPDKM determines K initial centroid by using sorting and linear systematic sampling heuristics.FindingsThe performance of the proposed algorithm is compared with KM, KM++, Var-Part, Murat-KM, Mean-KM and Sort-KM algorithms through Davies Bouldin score, Silhouette coefficient, SD Validity, S_Dbw Validity, Number of Iterations and CPU time validation indices on eight real business datasets. The experimental evaluation demonstrates that the NDPDKM algorithm reduces iterations, local optima, computing costs, and improves cluster performance, effectiveness, efficiency with stable convergence as compared to other algorithms. The NDPDKM algorithm minimizes the average computing time up to 34.83%, 90.28%, 71.83%, 92.67%, 69.53% and 76.03%, and reduces the average iterations up to 40.32%, 44.06%, 32.02%, 62.78%, 19.07% and 36.74% with reference to KM, KM++, Var-Part, Murat-KM, Mean-KM and Sort-KM algorithms.Originality/valueThe KM algorithm is the most widely used partitional clustering approach in data mining techniques that extract hidden knowledge, patterns and trends for decision-making strategies in business data. Business analytics is one of the applications of big data clustering where KM clustering is useful for the various subcategories of business analytics such as customer segmentation analysis, employee salary and performance analysis, document searching, delivery optimization, discount and offer analysis, chaplain management, manufacturing analysis, productivity analysis, specialized employee and investor searching and other decision-making strategies in business.

show abstract

“…Some authors (Fahad et al. , 2014; Pandey et al. , 2020) defined the clustering algorithm characteristics for big data using volume, variety and velocity characteristics.…”

Section: Related Work Of Initialization Methodsmentioning

confidence: 99%

Section: Analysis Of Existing Initialization Algorithm For Big Data C...mentioning

confidence: 99%

NDPD: an improved initial centroid method of partitional clustering for big data mining

Pandey

Shukla

2022

JAMR

Self Cite

View full text Add to dashboard Cite

show abstract

“…The volume, variety and velocity characteristics were utilized by the literature 2,69,70 to define the clustering algorithm characteristics for big data. The volume characteristic dictates the size, dimensionality and outliers of the data.…”

Section: Related Workmentioning

confidence: 99%

Min‐max kurtosis stratum mean: An improved K‐means cluster initialization approach for microarray gene clustering on multidimensional big data

Pandey

Shukla

2022

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

SUMMARY Microarray gene clustering is a big data application that employs the K‐means (KM) clustering algorithm to identify hidden patterns, evolutionary relationships, unknown functions and gene trends for disease diagnosis, tissue detection and biological analysis. The selection of initial centroids is a major issue in the KM algorithm because it influences the effectiveness, efficiency and local optima of the cluster. The existing initial centroid initialization algorithm is computationally expensive and degrades cluster quality due to the large dimensionality and interconnectedness of microarray gene data. To deal with this issue, this study proposed the min‐max kurtosis stratum mean (MKSM) algorithm for big data clustering in a single machine environment. The MKSM algorithm uses kurtosis for dimension selection, mean distance for gene relationship identification, and stratification for heterogeneous centroid extraction. The results of the presented algorithm are compared to the state‐of‐the‐art initialization strategy on twelve microarray gene datasets utilizing internal, external and statistical assessment criteria. The experimental results demonstrate that the MKSMKM algorithm reduces iterations, distance computation, data comparison and local optima, and improves cluster performance, effectiveness and efficiency with stable convergence.

show abstract

“…Janghel et al (Pandey et al, 2020) focused on correct diagnosing of breast cancer from the UCI repository dataset, by employing thirteen machine learning models and comparing them to the various measures. The results showed that AdaBoost, LR, and KNN models were promising high accuracy of 98% in experimenting with all the models.…”

Section: Breast Cancer Diseasementioning

confidence: 99%

Performance Evaluation of Different Machine Learning Classification Algorithms for Disease Diagnosis

Al-Hashem

Alqudah

Qananwah

2021

International Journal of E-Health and Medical Communications

View full text Add to dashboard Cite

Knowledge extraction within a healthcare field is a very challenging task since we are having many problems such as noise and imbalanced datasets. They are obtained from clinical studies where uncertainty and variability are popular. Lately, a wide number of machine learning algorithms are considered and evaluated to check their validity of being used in the medical field. Usually, the classification algorithms are compared against medical experts who are specialized in certain disease diagnoses and provide an effective methodological evaluation of classifiers by applying performance metrics. The performance metrics contain four criteria: accuracy, sensitivity, and specificity forming the confusion matrix of each used algorithm. We have utilized eight different well-known machine learning algorithms to evaluate their performances in six different medical datasets. Based on the experimental results we conclude that the XGBoost and K-Nearest Neighbor classifiers were the best overall among the used datasets and signs can be used for diagnosing various diseases.

show abstract

A Comprehensive Study of Clustering Algorithms for Big Data Mining with MapReduce Capability

Cited by 7 publications

References 22 publications

NDPD: an improved initial centroid method of partitional clustering for big data mining

NDPD: an improved initial centroid method of partitional clustering for big data mining

Min‐max kurtosis stratum mean: An improved K‐means cluster initialization approach for microarray gene clustering on multidimensional big data

Performance Evaluation of Different Machine Learning Classification Algorithms for Disease Diagnosis

Contact Info

Product

Resources

About