GUICOP: Specification-Based GUI Testing

Zaraket, Fadi A.; Masri, Wes; Adam, Marc T. P.; Hammoud, Dalal; Hamzeh, Raghd; Farhat, Raja; Khamissi, Elie; Noujaim, Joseph

doi:10.1109/icst.2012.168

Cited by 14 publications

(8 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…invoking clicks, to perform interaction during playback. However, both of these techniques suffer from different limitations that affect their robustness, usability, cost, but foremost their maintainability, suggested by empirical evidence related to the technique [Memon and Soffa(2003), Zaraket et al(2012) Zaraket, Masri, Adam, Hammoud, Hamzeh, Farhat, Khamissi, and Noujaim, Finsterwalder(2001), Li and Wu(2004), Grechanik et al(2009b) Grechanik, Xie, and Fu, Horowitz and Singhera(1993)]. The coordinate-based R&R techniques are sensitive to GUI layout change [Memon(2002)], e.g.…”

Section: Background and Related Workmentioning

confidence: 99%

Visual GUI testing in practice: challenges, problemsand limitations

2014

View full text Add to dashboard Cite

In today's software development industry, high-level tests such as Graphical User Interface (GUI) based system and acceptance tests are mostly performed with manual practices that are often costly, tedious and error prone. Test automation has been proposed to solve these problems but most automation techniques approach testing from a lower level of system abstraction. Their suitability for high-level tests has therefore been questioned. High-level test automation techniques such as Record and Replay exist, but studies suggest that these techniques suffer from limitations, e.g. sensitivity to GUI layout or code changes, system implementation dependencies, etc.Visual GUI Testing (VGT) is an emerging technique in industrial practice with perceived higher flexibility and robustness to certain GUI changes than previous high-level (GUI) test automation techniques. The core of VGT is image recognition which is applied to analyze and interact with the bitmap layer of a system's front end. By coupling image recognition with test scripts, VGT tools can emulate end user behavior on almost any GUI-based system, regardless of implementation language, operating system or platform. However, VGT is not without its own challenges, problems and limitations (CPLs) but, like for many other automated test techniques, there is a lack of empirically-based knowledge of these CPLs and how they impact industrial applicability. Crucially, there is also a lack of information on the cost of applying this type of test automation in industry.This manuscript reports an empirical, multi-unit case study performed at two Swedish companies that develop safety-critical software. It studies their transition from manual system test cases into tests automated with VGT. In total, four different test suites that together include more than 300 high-level system test cases were automated for two multi-million lines of code systems. The results show that the transitioned test cases could find defects in the tested systems and that all applicable test cases could be automated. However, during these transition projects a number of hurdles had to be addressed; a total of 58 different CPLs were identified and then categorized into 26 types. We present these CPL types and an analysis of the implications for the transition to and use of VGT in industrial software development practice. In addition, four high-level solutions are presented that were identified during the study, which would address about half of the identified CPLs.Furthermore, collected metrics on cost and return on investment of the VGT transition are reported together with information about the VGT suites' defect finding ability. Nine of the identified defects are reported, 5 of which were unknown to testers with extensive experience from using the manual test suites. The main conclusion from this study is that even though there are many challenges related to the transition and usage of VGT, the technique is still valuable, flexible and considered cost-effective by the industrial practit...

show abstract

Section: Background and Related Workmentioning

confidence: 99%

Visual GUI testing in practice: challenges, problemsand limitations

2014

View full text Add to dashboard Cite

show abstract

“…Other JUnit extensions that enable the user to write assertions also suffer from that problem, namely, JFCUnit (jfcunit.sourceforge.net), Pounder (pounder.sourceforge.net), Marathon (marathontesting.com), SWTBot (swtbot.org), UISpec4J (www.uispec4j.org), and Jemmy (jemmy.java.net). This paper is an extension of the authors' work presented in [36] and described in the related work section.…”

Section: Introductionmentioning

confidence: 94%

“… The most up-to-date implementation of GUICop supports the Java Swing graphics library, noting that our prior implementation [36] supported the C++ Qt library.…”

Section: This Paper Makes the Following Contributionsmentioning

confidence: 99%

GUICop: Approach and toolset for specification‐based GUI testing

Hammoud

Zaraket

Masri

2017

Software Testing Verif & Rel

Self Cite

View full text Add to dashboard Cite

Oracles used for testing graphical user interface (GUI) programs are required to take into consideration complicating factors such as variations in screen resolution or color scheme when comparing observed GUI elements to expected GUI elements.Researchers proposed fuzzy comparison rules and computationally expensive image processing techniques to tame the comparison process since otherwise the naïve matching comparison would be too constraining and consequently impractical.Alternatively, this paper proposes GUICop, a novel approach with a supporting toolset that takes (1) a GUI program and (2) user-defined GUI specifications characterizing the rendering behavior of the GUI elements, and checks whether the execution traces of the program satisfy the specifications. GUICop comprises the following: 1) a GUI Specification Language; 2) a Driver; 3)Instrumented GUI Libraries; 4) a Solver; and 5) a Code Weaver. The user defines the specifications of the subject GUI program using the GUI Specification Language.The Driver traverses the GUI structure of the program and generates events that drive its execution. The Instrumented GUI Libraries capture the GUI execution trace, i.e., information about the positions and visibility of the GUI elements. And the Solver, enabled by code injected by the Code Weaver, checks whether the traces satisfy the specifications.GUICop was successfully evaluated using four open source GUI applications that included eight defects, namely, Jajuk, Gason, JEdit, and TerpPaint.

show abstract

“…For instance, Memon and Soffa presented a regression testing technique for the graphical user interface (GUI) of a desktop application [52], while Brooks and Memon showed how to leverage user interaction profiles to guide GUI testing [53]. Furthermore, Hammoud et al and Zaraket et al proposed GUICop [54,55], a GUI testing approach that utilizes a specification language for capturing information about the layout and appearance of GUI components. This enables functional test cases to tolerate differences in the execution environment in which they are run-for example a change of screen resolution-while also checking properties of the GUI itself, such as the appearance and relative positioning of certain elements.…”

Section: Maximummentioning

confidence: 99%

Automated visual classification of DOM‐based presentation failure reports for responsive web pages

Althomali

Kapfhammer

McMinn

2021

Software Testing Verif & Rel

View full text Add to dashboard Cite

Since it is common for the users of a web page to access it through a wide variety of devices-including desktops, laptops, tablets and phones-web developers rely on responsive web design (RWD) principles and frameworks to create sites that are useful on all devices. A correctly implemented responsive web page adjusts its layout according to the viewport width of the device in use, thereby ensuring that its design suitably features the content. Since the use of complex RWD frameworks often leads to web pages with hardto-detect responsive layout failures (RLFs), developers employ testing tools that generate reports of potential RLFs. Since testing tools for responsive web pages, like ReDECHECK, analyse a web page representation called the Document Object Model (DOM), they may inadvertently flag concerns that are not human visible, thereby requiring developers to manually confirm and classify each potential RLF as a true positive (TP), false positive (FP), or non-observable issue (NOI)-a process that is time consuming and error prone. The conference version of this paper presented VISER, a tool that automatically classified three types of RLFs reported by ReDECHECK. Since VISER was not designed to automatically confirm and classify two types of RLFs that ReDECHECK's DOM-based analysis could surface, this paper introduces VERVE, a tool that automatically classifies all RLF types reported by ReDECHECK. Along with manipulating the opacity of HTML elements in a web page, as does VISER, the VERVE tool also uses histogram-based image comparison to classify RLFs in web pages. Incorporating both the 25 web pages used in prior experiments and 20 new pages not previously considered, this paper's empirical study reveals that VERVE's classification of all five types of RLFs frequently agrees with classifications produced manually by humans. The experiments also reveal that VERVE took on average about 4 s to classify any of the RLFs among the 469 reported by ReDECHECK. Since this paper demonstrates that classifying an RLF as a TP, FP, or NOI with VERVE, a publicly available tool, is less subjective and error prone than the same manual process done by a human web developer, we argue that it is well-suited for supporting the testing of complex responsive web pages.

show abstract

GUICOP: Specification-Based GUI Testing

Cited by 14 publications

References 14 publications

Visual GUI testing in practice: challenges, problemsand limitations

Visual GUI testing in practice: challenges, problemsand limitations

GUICop: Approach and toolset for specification‐based GUI testing

Automated visual classification of DOM‐based presentation failure reports for responsive web pages

Contact Info

Product

Resources

About