Automated GUI performance testing

Abstract

A significant body of prior work has devised approaches for automating the functional testing of interactive applications. However, little work exists for automatically testing their performance. Performance testing imposes additional requirements upon GUI test automation tools: the tools have to be able to replay complex interactive sessions, and they have to avoid perturbing the application's performance. We study the feasibility of using five Java GUI capture and replay tools for GUI performance test automation. Besides confirming the severity of the previously known GUI element identification problem, we also describe a related problem, the temporal synchronization problem, which is of increasing importance for GUI applications that use timer-driven activity. We find that most of the tools we study have severe limitations when used for recording and replaying realistic sessions of real-world Java applications and that all of them suffer from the temporal synchronization problem. However, we find that the most reliable tool, Pounder, causes only limited perturbation and thus can be used to automate performance testing. Based on an investigation of Pounder's approach, we further improve its robustness and reduce its perturbation. Finally, we demonstrate in a set of case studies that the conclusions about perceptible performance drawn from manual tests still hold when using automated tests driven by Pounder. Besides the significance of our findings to GUI performance testing, the results are also relevant to capture and replay-based functional GUI test automation approaches. This article is an extended version of our AST 2010 paper Jovic et al. (2010). A. Adamoli (&) Á D. Zaparanuks Á M. Jovic Á M. Hauswirth

A Survey on Load Testing of Large-Scale Software Systems

“…Examples of performance related aspects include response time, throughput and resource utilizations [24], [25], [74]. Performance testing can focus on parts of the system (e.g., unit performance testing [61] or GUI performance testing [75]), or on the overall system [24], [25], [48]. Performance testing can also study the efficiency of various design/architectural decisions [63], [64], [65], different algorithms [58], [59] and various system configurations [48], [71], [72].…”

EventBreak

“…The algorithm computes the Algorithm 1 Infer potential slowdown pairs. Input: Event-cost history h Output: Set S of potential slowdown pairs 1 …”

“…1 The administrative interface of a Joomla-based web site allows for adding menu items to the site, and it provides a way to set properties of entire menus. The event handler triggered when changing the properties of a menu validates the names of all menu items every time it is called.…”

JAutomate: A Tool for System- and Acceptance-test Automation

“…These limitations can be mitigated using another commonly used technique in industry, i.e. record and replay (R&R) [4]- [6]. R&R is a tool-driven, two step, technique where user interaction with the SUT is first recorded in a script that can then be replayed automatically to perform regression testing.…”

“…To mitigate these problems, test automation has been proposed as the solution, i.e. unit testing [3], record and replay [4]- [6], etc. However, these automation techniques have different limitations that restrict their use for system-and acceptance test automation, or make the automated tests costly to maintain.…”