A general framework for comparing automatic testing techniques of Android mobile apps

Amalfitano, Domenico; Amatucci, Nicola; Memon, Atif M.; Tramontana, Porfirio; Fasolino, Anna Rita

doi:10.1016/j.jss.2016.12.017

Cited by 52 publications

(56 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Previous work has compared different techniques and tools for AIG (Choudhary et al, 2015;Amalfitano et al, 2017;Zeng et al, 2016). Choudhary et al have compared AIG testing tools in terms of ease of use, ability to work on multiple platforms, code coverage, and ability to detect faults (Choudhary et al, 2015).…”

Section: Related Workmentioning

confidence: 99%

To the attention of mobile software developers: guess what, test your app!

Cruz

Abreu

2019

Empir Software Eng

View full text Add to dashboard Cite

Software testing is an important phase in the software development lifecycle because it helps in identifying bugs in a software system before it is shipped into the hand of its end users. There are numerous studies on how developers test general-purpose software applications. The idiosyncrasies of mobile software applications, however, set mobile apps apart from general-purpose systems (e.g., desktop, stand-alone applications, web services). This paper investigates working habits and challenges of mobile software developers with respect to testing. A key finding of our exhaustive study, using 1000 Android apps, demonstrates that mobile apps are still tested in a very ad hoc way, if tested at all. However, we show that, as in other types of software, testing increases the quality of apps (demonstrated in user ratings and number of code issues). Furthermore, we find evidence that tests are essential when it comes to engaging the community to contribute to mobile open source software. We discuss reasons and potential directions to address our findings. Yet another relevant finding of our study is that Continuous Integration and Continuous Deployment (CI/CD) pipelines are rare in the mobile apps world (only 26% of the apps are developed in projects employing CI/CD) -we argue that one of the main reasons is due to the lack of exhaustive and automatic testing.

show abstract

Section: Related Workmentioning

confidence: 99%

To the attention of mobile software developers: guess what, test your app!

Cruz

Abreu

2019

Empir Software Eng

View full text Add to dashboard Cite

show abstract

“…Amalfitano et al. compared testing techniques and tools for mobile apps and determined that the most widely used strategies are depth‐first and breath‐first search. In the present work, both strategies were implemented, but only the results of the depth‐first search are used in the evaluation section because the number of times that the app has to be initialized (or relaunched) is lower, which in turn decreases time required to extract the model.…”

Section: Implementation For Android Devicesmentioning

confidence: 99%

A formal approach to automatically analyse extra‐functional properties in mobile applications

Espada

Gallardo

Salmerón

et al. 2019

Software Testing Verif & Rel

View full text Add to dashboard Cite

Summary This paper presents an integrated approach for testing mobile applications (apps) against a set of extra‐functional properties to be used by app developers. The approach starts with the (manual or automatic) extraction of the interaction model, that is, a formal model of the potential user interactions with the app. The model is constructed to allow a model checking tool to exhaustively extract the so‐called app user flows, that is, the sequences of user actions, that constitute the test cases. In the final step, the app user flows are executed on the app running on real devices. The resulting execution traces are enriched with different measures and verified against a set of extra‐functional properties of interest. The approach has been adapted to analyse several applications running at the same time with several devices supporting the applications. This paper presents the definition and formalization of both the modelling language for the interaction model and the specification language to represent the extra‐functional properties. It also describes a methodology for automatically extracting the model. Finally, it presents an implementation focused on Android apps, which is integrated in the TRIANGLE testing framework, and the evaluation of the approach. © 2019 The Authors. Software Testing, Verification & Reliability Published by John Wiley & Sons Ltd.

show abstract

“…However, in fact, the GUI models of mobile apps may innately have an infinite number of UIs or may have infinite loops. As such, the GUI testing procedure may infinitely exercise app unless the termination condition is explicitly specified by the user . Hence, to address this problem, the existing tools set the termination condition for the testing procedure such as limited execution time, or manually predefined number of events to be injected, or the depth of exploration in the UI model.…”

Section: Proposed Approach: Mobolicmentioning

confidence: 99%

“…As such, the GUI testing procedure may infinitely exercise app unless the termination condition is explicitly specified by the user. 15 Hence, to address this problem, the existing tools set the termination condition for the testing procedure such as limited execution time, or manually predefined number of events to be injected, or the depth of exploration in the UI model. 14,15 Therefore, with such specified termination conditions, the existing tools may not discover certain app UIs, especially if they are located at deep levels in the UI model.Problem#2.…”

mentioning

confidence: 99%

Mobolic: An automated approach to exercising mobile application GUIs using symbiosis of online testing technique and customated input generation

et al. 2018

View full text Add to dashboard Cite

Summary The increasingly prevalent use of mobile devices has raised the popularity of mobile applications. Therefore, automated testing of mobile applications has become an extremely important task. However, it is still a challenge to automatically generate tests with high coverage for mobile applications due to their specific nontrivial structure and the highly interactive nature of graphical user interfaces (GUIs). In this paper, we propose a novel automated GUI testing technique for mobile applications, namely, Mobolic. In this approach, tests with high coverage are automatically generated and executed by combining the online testing technique and customated input generation. Employing the online testing technique, Mobolic systematically explores the app GUI without falling in a loop. It generates relevant events “on the fly” that are followed by an immediate execution. In addition, involving the customated input generation, Mobolic automatically generates relevant user inputs such as user‐predefined, concrete, or random ones. We implemented Mobolic and evaluated its performance on 10 real‐world open‐source Android applications. Our experimental results show the effectiveness and efficiency of Mobolic in terms of achieved code coverage and overall exercising time.

show abstract

A general framework for comparing automatic testing techniques of Android mobile apps

Cited by 52 publications

References 10 publications

To the attention of mobile software developers: guess what, test your app!

To the attention of mobile software developers: guess what, test your app!

A formal approach to automatically analyse extra‐functional properties in mobile applications

Mobolic: An automated approach to exercising mobile application GUIs using symbiosis of online testing technique and customated input generation

Contact Info

Product

Resources

About