Modeling and Extracting Load Intensity Profiles

Kistowski, Jóakim von; Herbst, Nikolas; Kounev, Samuel; Groenda, Henning; Stier, Christian; Lehrig, Sebastian

doi:10.1145/3019596

Cited by 13 publications

(11 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We also plan to implement direct transformations from the DSL to a scalable event-oriented discrete-event simulation as we are reaching the limit for simulating continuous sources (data streams). Finally, we will extend the specification of continuous data sources to include load intensity profiles that model variations in arrival rates [44].…”

Section: Discussionmentioning

confidence: 99%

PerTract: Model Extraction and Specification of Big Data Systems for Performance Prediction by the Example of Apache Spark and Hadoop

Kroß

Krcmar

2019

BDCC

View full text Add to dashboard Cite

Evaluating and predicting the performance of big data applications are required to efficiently size capacities and manage operations. Gaining profound insights into the system architecture, dependencies of components, resource demands, and configurations cause difficulties to engineers. To address these challenges, this paper presents an approach to automatically extract and transform system specifications to predict the performance of applications. It consists of three components. First, a system-and tool-agnostic domain-specific language (DSL) allows the modeling of performance-relevant factors of big data applications, computing resources, and data workload. Second, DSL instances are automatically extracted from monitored measurements of Apache Spark and Apache Hadoop (i.e., YARN and HDFS) systems. Third, these instances are transformed to model- and simulation-based performance evaluation tools to allow predictions. By adapting DSL instances, our approach enables engineers to predict the performance of applications for different scenarios such as changing data input and resources. We evaluate our approach by predicting the performance of linear regression and random forest applications of the HiBench benchmark suite. Simulation results of adjusted DSL instances compared to measurement results show accurate predictions errors below 15% based upon averages for response times and resource utilization.

show abstract

Section: Discussionmentioning

confidence: 99%

PerTract: Model Extraction and Specification of Big Data Systems for Performance Prediction by the Example of Apache Spark and Hadoop

Kroß

Krcmar

2019

BDCC

View full text Add to dashboard Cite

show abstract

“…The runtime model thus can only be used to evaluate how runtime management algorithms would perform under stable load conditions. [12] sketches an algorithm for the reconstruction of black-box resource demand functions from a series of load measurements. This paper contributes a novel model extraction approach that supports the reconstruction of timeline-based workload models from historical measurements.…”

Section: Methodsmentioning

confidence: 99%

“…It leverages [21] to gather basic infrastructure information, i.e., on the available servers. Our model extraction approach applies the algorithm from [12] to reconstruct workload models for individual VMs.…”

Section: Methodsmentioning

confidence: 99%

Rapid Testing of IaaS Resource Management Algorithms via Cloud Middleware Simulation

Stier

Domaschka

Koziolek

et al. 2018

Proceedings of the 2018 ACM/SPEC International Conference on Performance Engineering

Self Cite

View full text Add to dashboard Cite

Infrastructure as a Service (IaaS) Cloud services allow users to deploy distributed applications in a virtualized environment without having to customize their applications to a specific Platform as a Service (PaaS) stack. It is common practice to host multiple Virtual Machines (VMs) on the same server to save resources. Traditionally, IaaS data center management required manual effort for optimization, e.g. by consolidating VM placement based on changes in usage patterns. Many resource management algorithms and frameworks have been developed to automate this process. Resource management algorithms are typically tested via experimentation or using simulation. The main drawback of both approaches is the high effort required to conduct the testing. Existing Cloud or IaaS simulators require the algorithm engineer to reimplement their algorithm against the simulator's API. Furthermore, the engineer manually needs to define the workload model used for algorithm testing. We propose an approach for the simulative analysis of IaaS Cloud infrastructure that allows algorithm engineers and data center operators to evaluate optimization algorithms without investing additional effort to reimplement them in a simulation environment. By leveraging runtime monitoring data, we automatically construct the simulation models used to test the algorithms. Our validation shows that algorithm tests conducted using our IaaS Cloud simulator match the measured behavior on actual hardware.

show abstract

“…In order to identify the best cloud hosting option and choose virtual machine instances of the right size, it is essential to determine application computing need variability and resource usage levels under a specific workload. There have been several attempts to categorise cloud application workloads, such as [46] and [47]. However, this job is most thoroughly covered in [48,49], where groups of typical application workloads deployed in cloud environments are identified.…”

Section: Related Workmentioning

confidence: 99%

Model-Based Cloud Service Deployment Optimisation Method For Minimisation of Application Service Operational Cost

Stupar

Huljenić

2021

Preprint

View full text Add to dashboard Cite

Many of the currently existing solutions for cloud cost optimisation are aimed at cloud infrastructure providers, and they often deal only with specific types of application services, leaving the providers of cloud applications without the suitable cost optimization solution, especially concerning the wide range of different application types. In this paper, we present an approach that aims to provide an optimisation solution for the providers of applications hosted in the cloud environments, applicable at the early phase of a cloud application lifecycle and for a wide range of application services. The focus of this research is development of the method for identifying optimised service deployment option in available cloud environments based on the model of the service and its context, with the aim of minimising the operational cost of the cloud service, while fulfilling the requirements defined by the service level agreement. A cloud application context metamodel is proposed that includes parameters related to both the application service and the cloud infrastructure relevant for the cost and quality of service. By using the proposed optimisation method, the knowledge is gained about the effects that the cloud application context parameters have on the service cost and quality of service, which is then used to determine the optimised service deployment option. The service models are validated using cloud application services deployed in laboratory conditions, and the optimisation method is validated using the simulations based on proposed cloud application context metamodel. The experimental results based on two cloud application services demonstrate the ability of the proposed approach to provide relevant information about the impact of cloud application context parameters on service cost and quality of service, and use this information in the optimisation aimed at reducing service operational cost while preserving the acceptable service quality level. The results indicate the applicability and relevance of the proposed approach for cloud applications in the early service lifecycle phase since application providers can gain useful insights regarding service deployment decision without acquiring extensive datasets for the analysis.

show abstract

Modeling and Extracting Load Intensity Profiles

Cited by 13 publications

References 23 publications

PerTract: Model Extraction and Specification of Big Data Systems for Performance Prediction by the Example of Apache Spark and Hadoop

PerTract: Model Extraction and Specification of Big Data Systems for Performance Prediction by the Example of Apache Spark and Hadoop

Rapid Testing of IaaS Resource Management Algorithms via Cloud Middleware Simulation

Model-Based Cloud Service Deployment Optimisation Method For Minimisation of Application Service Operational Cost

Contact Info

Product

Resources

About