Automated, Cost-effective, and Update-driven App Testing

Abstract

Apps’ pervasive role in our society led to the definition of test automation approaches to ensure their dependability. However, state-of-the-art approaches tend to generate large numbers of test inputs and are unlikely to achieve more than 50% method coverage. In this article, we propose a strategy to achieve significantly higher coverage of the code affected by updates with a much smaller number of test inputs, thus alleviating the test oracle problem. More specifically, we present ATUA, a model-based approach that synthesizes App models with static analysis, integrates a dynamically refined state abstraction function and combines complementary testing strategies, including (1) coverage of the model structure, (2) coverage of the App code, (3) random exploration, and (4) coverage of dependencies identified through information retrieval. Its model-based strategy enables ATUA to generate a small set of inputs that exercise only the code affected by the updates. In turn, this makes common test oracle solutions more cost-effective, as they tend to involve human effort. A large empirical evaluation, conducted with 72 App versions belonging to nine popular Android Apps, has shown that ATUA is more effective and less effort-intensive than state-of-the-art approaches when testing App updates.

A Fast Crash Reproduction Method for Android Applications Based on Widget Hierarchy Graphs

“…In the experiments, we compare CrPDroid with PUMA [14], APE [13], ReCDroid [8] and ReproBot [10]. PUMA and APE are commonly used Android automated testing tools and are used as baselines in comparative experiments of related studies, such as Humanoid [26] and ATUA [27]. The goal of ReCDroid and ReproBot is similar to CrPDroid, which is to reproduce the crashes described in the bug reports.…”

“…During testing, the method updates the model dynamically to increase the probability of an event that rarely or never occurs when users use the application. ATUA [27] uses the application model to guide the dynamic exploration of the application under test, thereby achieving significantly higher coverage of the code affected by updates with a much smaller number of test inputs. The application model is initially created by static program analysis procedures and then refined during testing.…”