Categorization of Cloud Workload Types with Clustering

Orzechowski, Piotr; Proficz, Jerzy; Krawczyk, Henryk; Szymański, Julian

doi:10.1007/978-81-322-3592-7_31

Cited by 6 publications

(3 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Conventional clustering techniques, such as K-means and hierarchical clustering, have been extensively used for the process of cloud workload categorization [6,7,8,9,10,11,12,13]. These algorithms categorize workloads into specific grouping patterns based on similarity metrics.…”

Section: Related Workmentioning

confidence: 99%

An Ensemble Clustering Approach for Modeling Hidden Categorization Perspectives for Cloud Workloads

Daraghmeh¹,

Agarwal²,

Jararweh

2023

Preprint

View full text Add to dashboard Cite

Effectively managing cloud resources is a complex task due to the interdependencies of various cloud-hosted services and applications. This task is integral to workload categorization, which groups similar cloud workloads to inform workload scheduling and resource management procedures. Although traditional clustering algorithms can categorize workloads into single data grouping patterns, the multifaceted nature of cloud workloads may conceal multiple valid categorization perspectives, indicating a need for a more adaptable clustering approach. This paper presents a novel ensemble clustering approach to enhance the scheduling workload categorization process in cloud computing. Our approach combines various normalization and transformation techniques, including principal component analysis, to form multiple data preprocessing pipelines. The data derived from these pipelines then serve as input for multiple base clustering learners. A novel combined score based on the Silhouette score, Calinski-Harabasz index, and Davies-Bouldin index is then employed to select the optimal models and preprocessing setups. The clustering outcomes of these models are encoded and inputted into a meta-clustering algorithm, effectively capturing complex categorization perspectives. Evaluation of this approach using real-world workload trace data from Microsoft Azure significantly enhances workload segmentation efficacy, thereby improving resource management and quality of service in cloud data centers. This method offers a promising pathway toward improved workload clustering in cloud computing, demonstrating the practical utility of advanced ensemble clustering techniques.

show abstract

Section: Related Workmentioning

confidence: 99%

An Ensemble Clustering Approach for Modeling Hidden Categorization Perspectives for Cloud Workloads

Daraghmeh¹,

Agarwal²,

Jararweh

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…• analysis of the trade-off to find out potential points where values for measures incorporating execution time and energy used would be optimal for a specific application, • benchmarking other applications, especially those that take more power from our testbed systems, • power-aware modeling of compute devices in frameworks for simulation of application runs in high performance computing environments such as MERPSYS [23], • development of a tool for automatic detection of the optimal power settings for the aforementioned time-energy measures using historical data (e.g. via machine learning), • proposing a new method for minimizing the electrical energy usage dynamically at runtime for various HPC/cloud workloads [24]. We assume that the expectations of the IT industry will generate a high demand for green computing methods used for exchanging time of computations into savings in the energy consumption (e.g.…”

Section: Final Remarks and Future Workmentioning

confidence: 99%

Analyzing energy/performance trade-offs with power capping for parallel applications on modern multi and many core processors

Krzywaniak

Proficz

Czarnul

2018

Annals of Computer Science and Information Systems

Self Cite

View full text Add to dashboard Cite

In the paper we present extensive results from analyzing energy/performance trade-offs with power capping observed on four different modern CPUs, for three different parallel applications such as 2D heat distribution, numerical integration and Fast Fourier Transform. The CPU tested represent both multi-core type CPUs such as Intel R Xeon R E5, desktop and mobile i7 as well as many-core Intel R Xeon Phi TM x200 but also server, desktop and mobile solutions used widely nowadays. We show that using enforced power caps we can find points of lower than default energy consumption but mostly for desktop and mobile solutions at the cost of increased execution time. We show with particular numbers how energy consumed, power consumption and execution time change for the point of minimum energy used versus the default configuration with no power limit, for each application and each tested CPU.

show abstract

“…Workload [39] is defined as the resource consumption of a computational job that completed by a computational unit in a given time. The type of application significantly affects the resource allocation techniques [155]. This section summarizes five types of commonly encountered workload types in clouds.…”

Section: Workload Typesmentioning

confidence: 99%

An Evolutionary Computation Approach to Resource Allocation in Container-based Clouds

Tan¹

View full text Add to dashboard Cite

<p>A container-based cloud is a new trend in cloud computing that introduces more granular management of cloud resources. Compared with VM-based clouds, container-based clouds can further improve energy efficiency with a finer granularity resource allocation in data centers. The current allocation approaches for VM-based clouds cannot be used in container-based clouds. The first reason is that existing research lacks appropriate models that can represent the interaction of allocation features. Many critical features, such as VM overhead, are also not considered in the current models. The second reason is that current allocation approaches do not perform well to the three frequently encountered allocation scenarios, offline allocation, on-line allocation, and multi-objective allocation. Current approaches for these scenarios are mostly based on greedy heuristics that can be easily stuck at local optimum, or meta-heuristics that consider a simplified one-level allocation problem. Evolutionary Computation (EC) is particularly good at solving combinatorial optimization problems for both off-line and on-line scenarios. The overall goal of this thesis is to propose an EC approach to the three allocation scenarios in order to improve the performance of container-based clouds. Specifically, we aim to optimize energy consumption in all the scenarios. An additional objective, availability of the application, is considered for the multi-objective scenario. First, this thesis investigates two promising representations, vector-based and group-based. We propose two novel vector-based (e.g., Single- chromosome Genetic Algorithm (SGA) and Dual-chromosome GA (DGA)) and a group-based GA approaches for the off-line allocation scenario. Cor- responding genetic operators and decoding processes are also developed and evaluated. Two contributions have been made. Firstly, a novel offline model has been proposed based on current models with additional features. It can be used to evaluate allocation algorithms. Secondly, two types of problem representation, vector-based and group-based, are investigated and three novel approaches are proposed. The three approaches are compared with state-of-the-art approaches. The results show that all solutions produced by these approaches are better than the state-of-the-art approaches and group-based GA is the best approach. Second, this thesis proposes a novel genetic programming hyper-heuristic (GPHH) and a cooperative coevolution (CCGP)-based approach for the on-line allocation scenario. These hyper-heuristic methods can automatically generate allocation rules. For the GPHH-based approach, we develop a novel training procedure to generate reservation-based rules for allocating containers to VM instances. For the CCGP-based approach, we introduce a new terminal set and develop a training procedure to generate allocation rules for two-level allocations. We analyze both human-designed rules and generated rules to provide insights for algorithm designers. Two contributions have been proposed for the on-line problem. First, the on-line model for the on-line allocation scenario is developed. Second, a novel terminal set and training procedures are developed. The automatically generated heuristics perform significantly better than the manually designed heuristics. Third, this thesis proposes a multi-objective approach that generates a set of trade-off solutions for the cloud providers to choose from. Our novel approach, namely Nondominated Sorting-Group GA (NS-GGA), combines the group-based representation and the NSGA-II framework. The experimental results are compared with existing approaches. The results show that our proposed NS-GGA approach outperforms all other approaches. We propose two novelties. The first novelty is the multi-objective model including objectives of energy consumption and availability. The second novelty is the NS-GGA approach that combines the group-based representation with NSGA-II. The allocation solutions found by NS-GGA dominate the solutions found by other existing approaches.</p>

show abstract

Categorization of Cloud Workload Types with Clustering

Cited by 6 publications

References 13 publications

An Ensemble Clustering Approach for Modeling Hidden Categorization Perspectives for Cloud Workloads

An Ensemble Clustering Approach for Modeling Hidden Categorization Perspectives for Cloud Workloads

Analyzing energy/performance trade-offs with power capping for parallel applications on modern multi and many core processors

An Evolutionary Computation Approach to Resource Allocation in Container-based Clouds

Contact Info

Product

Resources

About