DroidbotX: Test Case Generation Tool for Android Applications Using Q-Learning

Abstract

Android applications provide benefits to mobile phone users by offering operative functionalities and interactive user interfaces. However, application crashes give users an unsatisfactory experience, and negatively impact the application’s overall rating. Android application crashes can be avoided through intensive and extensive testing. In the related literature, the graphical user interface (GUI) test generation tools focus on generating tests and exploring application functions using different approaches. Such tools must choose not only which user interface element to interact with, but also which type of action to be performed, in order to increase the percentage of code coverage and to detect faults with a limited time budget. However, a common limitation in the tools is the low code coverage because of their inability to find the right combination of actions that can drive the application into new and important states. A Q-Learning-based test coverage approach developed in DroidbotX was proposed to generate GUI test cases for Android applications to maximize instruction coverage, method coverage, and activity coverage. The overall performance of the proposed solution was compared to five state-of-the-art test generation tools on 30 Android applications. The DroidbotX test coverage approach achieved 51.5% accuracy for instruction coverage, 57% for method coverage, and 86.5% for activity coverage. It triggered 18 crashes within the time limit and shortest event sequence length compared to the other tools. The results demonstrated that the adaptation of Q-Learning with upper confidence bound (UCB) exploration outperforms other existing state-of-the-art solutions.

Accelerating Finite State Machine-Based Testing Using Reinforcement Learning

“…Recently, the use of RL has received attention in several areas of testing, including Android testing [59], mutation testing [60], online testing [61], and security testing [62], [63], [64]. Except for our past work [40] no approach has been proposed for FSM-based testing.…”

The integration of machine learning into automated test generation: A systematic mapping study

“…The main difference between publications lies in the reward function. Many base the reward on coverage of the states of the interface model (e.g., Yasin et al [69]), while incorporating additional information to bias state selection. Additional factors include magnitude of the state change [60], usage specifications [63,64], unique code functions called [65], curiosity-favouring exploration of new elements [66,67,70]-coverage of interaction methods (e.g., click, drag) [61], validity of the resulting state [67], and avoidance of navigation loops [59].…”

Fast Witness Generation for Readable GUI Test Scenarios via Generalized Experience Replay

“…Several GUI test generators including AutoBlackTest [52], AntQ [53], TESTAR [54], QBE [19], curiosity-driven Qtesting [23], DroidBotX [24], SQDroid [25], Deep-GUIT [26], and SARSA [27] use RL to guide the test generation. All of these test generators are exploratory tools because they aim to explore a given AUT quickly and as much as possible.…”

“…F-Droid is an open-source Android GUI application repository. Multiple Android GUI test generators in the literature [19], [24] use Android applications from this repository.…”