In-Test Adaptation of Workload in Enterprise Application Performance Testing

Kaczmarski, Maciej; Perry, Philip; Murphy, John; Portillo-Dominguez, A. Omar

doi:10.1145/3053600.3053614

Cited by 5 publications

(8 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Meanwhile, the work presented in [33] describes the performance analysis of software systems as a comparison of two versions of software and their performance results to find possible (regression) bugs. In response to these limitations, approaches that dynamically adapt the workload have been proposed [3,22]. This paper conducts an evaluation of DYNAMO because of its advantages of adjusting the workload on the fly using the analysis of key performance metrics to create a customized workload for the AUT with minimal knowledge or experience from the tester.…”

Section: Background and Related Workmentioning

confidence: 99%

Towards an Efficient Performance Testing Through Dynamic Workload Adaptation

Huerta-Guevara

Ayala-Rivera

Murphy

et al. 2019

Testing Software and Systems

Self Cite

View full text Add to dashboard Cite

Performance testing is a critical task to ensure an acceptable user experience with software systems, especially when there are high numbers of concurrent users. Selecting an appropriate test workload is a challenging and time-consuming process that relies heavily on the testers' expertise. Not only are workloads application-dependent, but also it is usually unclear how large a workload must be to expose any performance issues that exist in an application. Previous research has proposed to dynamically adapt the test workloads in real-time based on the application behavior. By reducing the need for the trial-and-error test cycles required when using static workloads, dynamic workload adaptation can reduce the effort and expertise needed to carry out performance testing. However, such approaches usually require testers to properly configure several parameters in order to be effective in identifying workloaddependent performance bugs, which may hinder their usability among practitioners. To address this issue, this paper examines the different criteria needed to conduct performance testing efficiently using dynamic workload adaptation. We present the results of comprehensively evaluating one such approach, providing insights into how to tune it properly in order to obtain better outcomes based on different scenarios. We also study the effects of varying its configuration and how this can affect the results obtained.

show abstract

Section: Background and Related Workmentioning

confidence: 99%

Towards an Efficient Performance Testing Through Dynamic Workload Adaptation

Huerta-Guevara

Ayala-Rivera

Murphy

et al. 2019

Testing Software and Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…(2) The second type of run used the preliminary version of DYNAMO based on heuristic policies derived from PetStore (referred as h-DYNAMO [15]). (3) The third type of run used the work proposed in this paper which adopts our new adaptive logic (DYNAMO).…”

Section: Log Outmentioning

confidence: 99%

“…To address this challenge, our research has focused on developing techniques that improve the identification of workload-dependent performance issues, as well as their root causes, in order to increase the productivity of testers (hereinafter referred as users) by reducing the effort and expertise required in this process. In a previous work [15], we proposed an automated approach which dynamically adapts the workload used by a testing tool. However, that preliminary version was based on heuristic policies derived from the studied AUT; hence, it was not practical for real-world usage (as it was not application-independent).…”

Section: Introductionmentioning

confidence: 99%

One Size Does Not Fit All

Ayala-Rivera

Kaczmarski

Murphy

et al. 2018

Proceedings of the 2018 ACM/SPEC International Conference on Performance Engineering

Self Cite

View full text Add to dashboard Cite

Publisher ACM Link to online version https://icpe2018.spec.org/ Item record/more information http://hdl.handle.net/10197/9960Publisher's statement ABSTRACT Carrying out proper performance testing is considerably challenging. In particular, the identification of performance issues, as well as their root causes, is a time-consuming and complex process which typically requires several iterations of tests (as this type of issues can depend on the input workloads), and heavily relies on human expert knowledge. To improve this process, this paper presents an automated approach (that extends some of our previous work) to dynamically adapt the workload (used by a performance testing tool) during the test runs. As a result, the performance issues of the tested application can be revealed more quickly; hence, identifying them with less effort and expertise. Our experimental evaluation has assessed the accuracy of the proposed approach and the time savings that it brings to testers. The results have demonstrated the benefits of the approach by achieving a significant decrease in the time invested in performance testing (without compromising the accuracy of the test results), while introducing a low overhead in the testing environment.

show abstract

“…Finally, other research works have aimed to reduce the expertise and effort needed to conduct useful testing. For example, by eliminating the need of manually configuring a diagnosis tool [20], or setting an appropriate test workload [21]. In contrast to these works, which have aimed to improve other aspects of the testing process, our tool has been designed with the aim of facilitating the creation of useful cluster test environments.…”

Section: B Related Workmentioning

confidence: 99%