Sidereal: Statistical adaptive generation of robust locators for web testing

Abstract

SummaryBy ensuring adequate functional coverage, End‐to‐End (E2E) testing is a key enabling factor of continuous integration. This is even more true for web applications, where automated E2E testing is the only way to exercise the full stack used to create a modern application. The test code used for web testing usually relies on DOM locators, often expressed as XPath expressions, to identify the web elements and to extract the data checked in assertions. When applications evolve, the most dominant cost for the evolution of test code is due to broken locators, which fail to locate the target element in the novel versions and must be repaired. In this paper, we formulate the robust XPath locator generation problem as a graph exploration problem, instead of relying on ad‐hoc heuristics as the one implemented by the state of the art tool robula+. Our approach is based on a statistical adaptive algorithm implemented by the tool sidereal, which outperforms robula+'s heuristics in terms of robustness by learning the potential fragility of HTML properties from previous versions of the application under test. sidereal was applied to six applications and to a total of 611 locators and was compared against two baseline algorithms, robula+ and Montoto. The adoption of sidereal results in a significant reduction of the number of broken locators (respectively ‐55% and ‐70%). The time for generating such robust locators was deemed acceptable being in the order of hundredths of second.

Similarity-based Web Element Localization for Robust Test Automation

“…SIDEREAL is a tool for automated end-to-end (E2E) testing of web applications [50]. It addresses the problem of broken locators by using a statistical adaptive algorithm that learns the potential fragility of web element properties to generate robust XPath locators.…”

An empirical study to compare three web test automation approaches: NLP‐based, programmable, and capture&replay

“…Moreover, adopting the PT approach allows the developer to choose the resilient locators [26][27][28] used to locate web elements, and this can help to write more robust test suites (although this can be done, to some extent, also in the CRT approach). Finally, the PT approach allows to adopt more powerful assertions than the other two E2E approaches: PT testing frameworks offer plenty of different assertion kinds, and they can even be extended using frameworks dedicated to this aspect (for example, Hamcrest and AssertJ in the Java language 29 ).…”

Fight silent horror unit test methods by consulting a TestWizard

“…A threat to the external validity of this study can be related to the use of students in the test methods implementation. Students are often employed in software engineering empirical studies (e.g., other works [26][27][28] ). We cannot expect students to perform as well as professionals, but we expect to observe similar trends.…”

“…The authors declare that they have no conflicts of interest. # As the two datasets refer to the same oracle and the same test plan, the instrumentation was unique for both k As an example, assume that the system requirement is a method yielding a random date and that the SUT is the approximate implementation building the date from random numbers in [1,28] for the day and in [1,12] for the month. Consider a test T verifying that the result's day is smaller than 30.…”