Using a pilot study to derive a GUI model for automated testing

Abstract

Graphical user interfaces (GUIs) are one of the most commonly used parts of today's software. Despite their ubiquity, testing GUIs for functional correctness remains an under-studied area. A typical GUI gives many degrees of freedom to an end-user, leading to an enormous input event interaction space that needs to be tested. GUI test designers generate and execute test cases (modeled as sequences of user events) to traverse its parts; targeting a sub-space in order to maximize fault detection is a non-trivial task. In this vein, in previous work, we used informal GUI code examination and personal intuition to develop an event-interaction graph (EIG). In this paper we empirically derive the EIG model via a pilot study, and the resulting EIG validates our intuition used in previous work; the empirical derivation process also allows for model evolution as our understanding of GUI faults improves. Results of the pilot study show that events interact in complex ways; a GUI's response to an event may vary depending on the context established by preceding events and their execution order. The EIG model helps testers to understand the nature of interactions between GUI events when executed in test cases and why certain events detect faults, so that they can better traverse the event space. New test adequacy criteria are defined for the EIG; new algorithms use these criteria and EIG to systematically generate test cases that are shown to be effective on four fielded open-source applications.

People-Centered Software Development: An Overview of Agile Methodologies

“…GUIs allow a great amount of freedom to user interaction, making them very complex. When testing the functionality of a GUI-based application using automated tests, two factors are of prime importance: the number of steps in the test; and the number of times each action in the GUI is taken within a test [17]. In Section 3.4, the implications of the complexity of modern GUIs will be explored in more detail.…”

“…Additionally, because of this cross-process testing, it's rare for all information about a widget to be exposed. For example, the Button class as implemented in Windows Presentation Foundation 16 can be tested through the InvokePattern interface in the Windows Automation API 17 . The Button itself has 136 properties, but InvokePattern exposes only 20 of these for use by test code.…”

Trends in Model-based GUI Testing

“…Similar to the EFG, an Event Interaction Graph [54,52,50] (EIG) is a directed graph where the nodes represent events. The EIG is generated from an EFG.…”

“…The purpose of an EIG is to find a compact representation of the GUI for a more efficient test case generation. In [52], the concept of EIG is motivated by distinguishing three different types of events:…”

“…However, the options in the menu, for example, Copy, Cut and Paste can impact on the system state and reveal a bug. In this way, the EIG is proposed [54,52,50]. The basic idea is that drop off events in the EFG will not directly change the system state.…”