A self-scalable load injection service

Abstract

20 pages - Online version to be included in an issueInternational audienceLoad testing of applications is an important and costly activity for software provider companies. Classical solutions are very difficult to set up statically, and their cost is prohibitive in terms of both human and hardware resources. Virtualized cloud computing platforms provide new opportunities for stressing an application's scalability, by providing a large range of flexible and less expensive (pay-per-use model) computation units. On the basis of these advantages, load testing solutions could be provided on demand in the cloud. This paper describes a Benchmark-as-a-Service solution that automatically scales the load injection platform and facilitates its setup according to load profiles. Our approach is based on: (i) virtualization of the benchmarking platform to create self-scaling injectors; (ii) online calibration to characterize the injector's capacity and impact on the benched application; and (iii) a provisioning solution to appropriately scale the load injection platform ahead of time. We also report experiments on a benchmark illustrating the benefits of this system in terms of cost and resource reductions. Copyright © 2013 John Wiley & Sons, Ltd

Self-scalable services in service oriented software for cost-effective data farming

“…Also, the set of load metrics is constrained to generic ones like response time, load, and request queue length. In [34], authors utilize clouds elasticity to implement a self-scalable service for stress testing, called Benchmark as a Service. The proposed solution automatically adjusts the number of virtual machines, which generate load onto a stressed service, based on a defined testing scenario and virtual machines capacity.…”

A Systematic Mapping Study of Empirical Studies on Software Cloud Testing Methods

“…The study [48] presents a benchmarking-as-a-service framework that automatically scales the injection load platform. Three experiment scenarios were performed, with two SUTs selected to test in these scenarios.…”

PEESOS-Cloud: A Workload-Aware Architecture for Performance Evaluation in Service-Oriented Systems

“…When the workload is executed centrally, the mischaracterisation caused by the presence of noise occurs as a result of problems such as the overloading of the components responsible for generating the workload, especially when a single machine has to represent many clients [9], [14]. [21] shows how the excessive number of threads created in the instances that carry out load generation have an adverse effect on the generation of the workload.…”

“…It is a complex task to detect the occurrence of failures during an experiment and quantify the effects these cause. The reason for this is that the losses incurred go beyond the question of defining the QoS levels of the service measured, or the extent of the wasted resources [9].…”