Efficient parallel spectral clustering algorithm design for large data sets under cloud computing environment

Jin, Ran; Kou, Chunhai; Liu, Ruijuan; Li, Yefeng

doi:10.1186/2192-113x-2-18

Cited by 19 publications

(8 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similarly, several real-time cloud data mining frameworks, algorithms, and services are available that provide information through a number of applications [44]. These real-time cloud data mining and load balancing applications are VM tasks classification for load balancing, feature extraction, anomaly, and intrusion detection, open shop scheduling, attribute importance, spatial classifications, data analysis and satellite imagery [45], spectral and statistical data analysis [46], gene expression data mining and bioinformatics [47], geo-spatial analysis and geoinformatics, large-scale mining in big data and web mining [48], machine-learning applications [49], high-dimensional data mining [50], highly diversified and dense data mining in rule mining [51], the security of data in the cloud, clustering, datacenters resources optimization in the cloud, noise removal, reactive power problem, face recognition, biomedical image processing, teaching based learning, manufacturing design, water resource problem and routing optimization. This research is mainly focused on classification, specifically the combination of classifier with a load balancer in the cloud.…”

Section: Literature Reviewmentioning

confidence: 99%

Modeling an Optimized Approach for Load Balancing in Cloud

et al. 2020

View full text Add to dashboard Cite

Despite significant infrastructure improvements, cloud computing still faces numerous challenges in terms of load balancing. Several techniques have been applied in the literature to improve load balancing efficiency. Recent research manifested that load balancing techniques based on metaheuristics provide better solutions for proper scheduling and allocation of resources in the cloud. However, most of the existing approaches consider only a single or few QoS metrics and ignore many important factors. The performance efficiency of these approaches is further enhanced by merging with machine learning techniques. These approaches combine the relative benefits of load balancing algorithm backed up by powerful machine learning models such as Support Vector Machines (SVM). In the cloud, data exists in huge volume and variety that requires extensive computations for its accessibility, and hence performance efficiency is a major concern. To address such concerns, we propose a load balancing algorithm, namely, Data Files Type Formatting (DFTF) that utilizes a modified version of Cat Swarm Optimization (CSO) along with SVM. First, the proposed system classifies data in the cloud from diverse sources into various types, such as text, images, video, and audio using one to many types of SVM classifiers. Then, the data is input to the modified load balancing algorithm CSO that efficiently distributes the load on VMs. Simulation results compared to existing approaches showed an improved performance in terms of throughput (7%), the response time (8.2%), migration time (13%), energy consumption (8.5%), optimization time (9.7%), overhead time (6.2%), SLA violation (8.9%), and average execution time (9%). These results outperformed some of the existing baselines used in this research such as CBSMKC, FSALB, PSO-BOOST, IACSO-SVM, CSO-DA, and GA-ACO.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Modeling an Optimized Approach for Load Balancing in Cloud

et al. 2020

View full text Add to dashboard Cite

show abstract

“…(2)Analysis of scalability According to the paper [Ran Jin,37], the formula is η=S p /N，wherein, S p represents the speedup ratio, N means the number of cluster nodes. Fig.…”

Section: (1)test Of Speedup Ratiomentioning

confidence: 99%

An Optimized Iterative Semantic Compression Algorithm And Parallel Processing for Large Scale Data

Jin¹,

Chen²,

Tung³

et al. 2018

KSII TIIS

View full text Add to dashboard Cite

With the continuous growth of data size and the use of compression technology, data reduction has great research value and practical significance. Aiming at the shortcomings of the existing semantic compression algorithm, this paper is based on the analysis of ItCompress algorithm, and designs a method of bidirectional order selection based on interval partitioning, which named An Optimized Iterative Semantic Compression Algorithm (Optimized ItCompress Algorithm). In order to further improve the speed of the algorithm, we propose a parallel optimization iterative semantic compression algorithm using GPU (POICAG) and an optimized iterative semantic compression algorithm using Spark (DOICAS). A lot of valid experiments are carried out on four kinds of datasets, which fully verified the efficiency of the proposed algorithm.

show abstract

“…Jin et al proposed a distributed version of the spectral clustering algorithm using the Map Reduce Hadoop framework . The authors parallelized the similarity matrix computation and the subsequent inference of the dominant eigenvalues from the similarity matrix.…”

Section: Related Workmentioning

confidence: 99%

A distributed parallel algorithm for inferring hierarchical groups from large‐scale text corpuses

Seshadri¹,

Mercy²,

Manohar³

2017

Concurrency and Computation

View full text Add to dashboard Cite

Summary We propose a distributed parallel algorithm for inferring the hierarchical groups present in a large‐scale text corpus. The algorithm is designed to deal with corpuses that typically do not fit into the main memory of a workstation computer. The key contribution of this paper lies in its proposal and verification of a parallel distributed algorithm that exploits the advantages of two complementary techniques based on (i) localized modularity optimization and (ii) spectral clustering. Based on our experimental observations, these are complementary in the sense that the former excels at finding coarse groups in a large‐scale network, while the latter demands a heavy memory footprint but is effective in inferring tightly knit fine‐grained groups. Empirical evaluation of the distributed implementation scheme shows that the algorithm exhibits a significant speed‐up when compared to existing algorithms like Louvain and, at the same time, produces better quality clusters than either Louvain or spectral clustering algorithms in terms of the F‐score and Rand index.

show abstract

Efficient parallel spectral clustering algorithm design for large data sets under cloud computing environment

Cited by 19 publications

References 8 publications

Modeling an Optimized Approach for Load Balancing in Cloud

Modeling an Optimized Approach for Load Balancing in Cloud

An Optimized Iterative Semantic Compression Algorithm And Parallel Processing for Large Scale Data

A distributed parallel algorithm for inferring hierarchical groups from large‐scale text corpuses

Contact Info

Product

Resources

About