Configurations in Android testing: they matter

Kowalczyk, Emily; Cohen, Myra B.; Memon, Atif M.

doi:10.1145/3243218.3243219

Cited by 17 publications

(3 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another line of work in this area aims to facilitate the testing of Android apps in different settings and/or contexts. Kowalczyk et al [27] observe that Android apps may exhibit distinct behaviors when run on different devices and operating systems with different internal settings, and they argue Android testing should be aware of the apps' internal and external configurations. Liang et al [31,32] focus on contextual parameters like device heterogeneity, wireless network speed, and unpredictable sensor input, and propose the contextual fuzzing technique that systematically explores a range of mobile contexts.…”

Section: Automated Android Testingmentioning

confidence: 99%

Preference-wise Testing of Android Apps via Test Amplification

Pan

Pei

et al. 2023

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

Preferences, the setting options provided by Android, are an essential part of Android apps. Preferences allow users to change app features and behaviors dynamically, and therefore their impacts need to be considered when testing the apps. Unfortunately, few test cases explicitly specify the assignments of valid values to the preferences, or configurations , under which they should be executed, and few existing mobile testing tools take the impact of preferences into account or provide help to testers in identifying and setting up the configurations for running the tests. This paper presents the Prefest approach to effective testing of Android apps with preferences. Given an Android app and a set of test cases for the app, Prefest amplifies the test cases with a small number of configurations to exercise more behaviors and detect more bugs that are related to preferences. In an experimental evaluation conducted on real-world Android apps, amplified test cases produced by Prefest from automatically generated test cases covered significantly more code of the apps and detected 7 real bugs, and the tool’s test amplification time was at the same order of magnitude as the running time of the input test cases. Prefest ’s effectiveness and efficiency in amplifying programmer-written test cases was comparable with that in amplifying automatically generated test cases.

show abstract

Section: Automated Android Testingmentioning

confidence: 99%

Preference-wise Testing of Android Apps via Test Amplification

Pan

Pei

et al. 2023

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

show abstract

“…We chose open-source apps in the wild as the subjects to evaluate Genie because we can directly communicate with app developers for bug confirmation. Specifically, we scrawled all the apps from F-Droid [F-Droid 2020], the largest open-source Android app market, and also collected the subjects from recent research on GUI testing of Android [Choudhary et al 2015;Fazzini et al 2018;Kowalczyk et al 2018;]. Among these apps, we selected the apps on GitHub, by which we can submit bug reports.…”

Section: Evaluation Setupmentioning

confidence: 99%

Fully automated functional fuzzing of Android apps for detecting non-crashing logic bugs

Yan

Wang

et al. 2021

Proc. ACM Program. Lang.

View full text Add to dashboard Cite

Android apps are GUI-based event-driven software and have become ubiquitous in recent years. Obviously, functional correctness is critical for an app’s success. However, in addition to crash bugs, non-crashing functional bugs (in short as “non-crashing bugs” in this work) like inadvertent function failures, silent user data lost and incorrect display information are prevalent, even in popular, well-tested apps. These non-crashing functional bugs are usually caused by program logic errors and manifest themselves on the graphic user interfaces (GUIs). In practice, such bugs pose significant challenges in effectively detecting them because (1) current practices heavily rely on expensive, small-scale manual validation ( the lack of automation ); and (2) modern fully automated testing has been limited to crash bugs ( the lack of test oracles ). This paper fills this gap by introducing independent view fuzzing , a novel, fully automated approach for detecting non-crashing functional bugs in Android apps. Inspired by metamorphic testing, our key insight is to leverage the commonly-held independent view property of Android apps to manufacture property-preserving mutant tests from a set of seed tests that validate certain app properties. The mutated tests help exercise the tested apps under additional, adverse conditions. Any property violations indicate likely functional bugs for further manual confirmation. We have realized our approach as an automated, end-to-end functional fuzzing tool, Genie. Given an app, (1) Genie automatically detects non-crashing bugs without requiring human-provided tests and oracles (thus fully automated ); and (2) the detected non-crashing bugs are diverse (thus general and not limited to specific functional properties ), which set Genie apart from prior work. We have evaluated Genie on 12 real-world Android apps and successfully uncovered 34 previously unknown non-crashing bugs in their latest releases — all have been confirmed, and 22 have already been fixed. Most of the detected bugs are nontrivial and have escaped developer (and user) testing for at least one year and affected many app releases, thus clearly demonstrating Genie’s effectiveness. According to our analysis, Genie achieves a reasonable true positive rate of 40.9%, while these 34 non-crashing bugs could not be detected by prior fully automated GUI testing tools (as our evaluation confirms). Thus, our work complements and enhances existing manual testing and fully automated testing for crash bugs.

show abstract

“…Other aspects have been shown to impact testing beyond those associated with fragmentation. These include orientation of the device (e.g., landscape or portrait), localization (which may load different resources), and permissions [60]. None of the tools tested in this experiment have considered these aspects explicitly.…”

Section: Fragmentationmentioning

confidence: 99%

An Empirical Analysis of Test Input Generation Tools for Android Apps through a Sequence of Events

et al. 2020

View full text Add to dashboard Cite

Graphical User Interface (GUI) testing of Android apps has gained considerable interest from the industries and research community due to its excellent capability to verify the operational requirements of GUI components. To date, most of the existing GUI testing tools for Android apps are capable of generating test inputs by using different approaches and improve the Android apps’ code coverage and fault detection performance. Many previous studies have evaluated the code coverage and crash detection performances of GUI testing tools in the literature. However, very few studies have investigated the effectiveness of the test input generation tools, especially in the events sequence length of the overall test coverage and crash detection. The event sequence length generally shows the number of steps required by the test input generation tools to detect a crash. It is critical to highlight its effectiveness due to its significant effects on time, testing effort, and computational cost. Thus, this study evaluated the effectiveness of six test input generation tools for Android apps that support the system events generation on 50 Android apps. The generation tools were evaluated and compared based on the activity coverage, method coverage, and capability in detecting crashes. Through a critical analysis of the results, this study identifies the diversity and similarity of test input generation tools for Android apps to provide a clear picture of the current state of the art. The results revealed that a long events sequence performed better than a shorter events sequence. However, a long events sequence led to a minor positive effect on the coverage and crash detection. Moreover, the study showed that the tools achieved less than 40% of the method coverage and 67% of the activity coverage.

show abstract

Configurations in Android testing: they matter

Cited by 17 publications

References 17 publications

Preference-wise Testing of Android Apps via Test Amplification

Preference-wise Testing of Android Apps via Test Amplification

Fully automated functional fuzzing of Android apps for detecting non-crashing logic bugs

An Empirical Analysis of Test Input Generation Tools for Android Apps through a Sequence of Events

Contact Info

Product

Resources

About