Scripted GUI testing of Android open-source apps: evolution of test code and fragility causes

Coppola, Riccardo; Morisio, Maurizio; Torchiano, Marco; Ardito, Luca

doi:10.1007/s10664-019-09722-9

Cited by 21 publications

(14 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These testing techniques give full control to the testers for exercising complex use cases against the app GUI, at the cost of high development and maintenance costs. Test cases written with Automation APIs are also significantly prone to fragility issues, meaning that they need relevant maintenance effort when the application is subject to typical maintenance [3]. Examples of Automation APIs and Frameworks are the official Espresso [11] and UI Automator [12] tools, Calabash [13], Ranorex [14], Robolectric [15] and Robotium [16].…”

Section: B Testing Strategies For Android Appsmentioning

confidence: 99%

“…We hence adopted the Word2Vec Word Embedding technique, in which each word is identified by a word vector in a vector space of 300 dimensions. We adopted the pre-trained word embedding model by Google News Word2Vec model as released by Mikolov et al 3 . The model pairs every word string found in the strings.…”

Section: ) Composing the Feature Vectormentioning

confidence: 99%

“…Several empirical works in literature have highlighted the amount of maintenance typically required by Android test suites [1]- [3]. In contrast, several survey-based studies have linked the intrinsic difficulties in test script development to the relatively low adoption of scripted testing frameworks among Android developers [4], [5].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Automated Test Selection for Android Apps Based on APK and Activity Classification

et al. 2020

Self Cite

View full text Add to dashboard Cite

Several techniques exist for mobile test automation, from script-based techniques to automated test generation based on GUI models. Most techniques fall short in being adopted extensively by practitioners because of the very costly definition (and maintenance) of test cases. We present a novel testing framework for Android apps that allows a developer to write effective test scripts without having to know the implementation details and the user interface of the app under test. The main goal of the framework is to generate adaptive tests that can be executed on a significant number of apps, or different releases of the same app, without manual editing of the tests. The frameworks consists of: (1) a Test Scripting Language, that allows the tester to write generic test scripts tailored to activity and app categories; (2) a State Graph Modeler, that creates a model of the app's GUI, identifying activities (i.e., screens) and widgets; (3) an app classifier that determines the type of application under test; (4) an activity classifier that determines the purpose of each screen; (5) a test adapter that executes test scripts that are compatible with the specific app and activity, automatically tailoring the test scripts to the classes of the app and the activities under test. We evaluated empirically the components of our testing framework. The classifiers were able to outperform available approaches in the literature. The developed testing framework was able to correctly adapt highlevel test cases to 28 out of 32 applications, and to reduce the LOCs of the test scripts of around 90%. We conclude that machine learning can be fruitfully applied to the creation of high-level, adaptive test cases for Android apps. Our framework is modular in nature and allows expansions through the addition of new commands to be executed on the classified apps and activities.

show abstract

Section: B Testing Strategies For Android Appsmentioning

confidence: 99%

Section: ) Composing the Feature Vectormentioning

confidence: 99%

See 1 more Smart Citation

Automated Test Selection for Android Apps Based on APK and Activity Classification

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…[11]; many studies focused on opinion mining from Play Store Reviews: Genc et al provide a systematic literature review of works leveraging such practice [12]. Other large-scale studies were based on all the Android applications whose source code was published on GitHub: in our previous work, we analyzed the scripted testing diffusion and evolution on a data set of 280,000 Android projects [13].…”

Section: Threats To Validitymentioning

confidence: 99%

Characterizing the transition to Kotlin of Android apps: a study on F-Droid, Play Store, and GitHub

Coppola

Ardito

Torchiano

2019

Proceedings of the 3rd ACM SIGSOFT International Workshop on App Market Analytics

Self Cite

View full text Add to dashboard Cite

Context: Kotlin is a novel language that represents an alternative to Java, and has been recently adopted as a first-class programming language for Android applications. Kotlin is achieving a significant diffusion among developers, and several studies have highlighted various advantages of the language when compared to Java.Goal: The objective of this paper is to analyze a set of open-source Android apps, to evaluate their transition to the Kotlin programming language throughout their lifespan and understand whether the adoption of Kotlin has impacts on the success of Android apps.Methods: We mined all the projects from the F-Droid repository of Android open-source applications, and we found the corresponding projects on the official Google Play Store and on the GitHub platform. We defined a set of eight metrics to quantify the relevance of Kotlin code in the latest update and through all releases of an application. Then, we statistically analyzed the correlation between the presence of Kotlin code in a project and popularity metrics mined from the platforms where the apps were released.Results: Of a set of 1232 projects that were updated after October 2017, near 20% adopted Kotlin and about 12% had more Kotlin code than Java; most of the projects that adopted Kotlin quickly transitioned from Java to the new language. The projects featuring Kotlin had on average higher popularity metrics; a statistically significant correlation has been found between the presence of Kotlin and the number of stars on the GitHub repository.Conclusion: The Kotlin language seems able to guarantee a seamless migration from Java for Android developers. With an inspection on a large set of open-source Android apps, we observed that the adoption of the Kotlin language is rapid (when compared to the average lifespan of an Android project) and seems to come at no cost in terms of popularity among the users and other developers.

show abstract

“…Among the main difficulties discouraging mobile testing, mobile apps are particularly prone to GUI testing Fragility, due to a rapid evolution of their GUI. A GUI test case used for regression testing is said to be fragile when failures are triggered not because of the injection of defects in the AUT, but for (even minor) changes in the GUI pictorial appearance or definition [12].…”

Section: Introduction and Related Workmentioning

confidence: 99%

Fragility of layout-based and visual GUI test scripts: an assessment study on a hybrid mobile application

Coppola

Ardito

Torchiano

2019

Proceedings of the 10th ACM SIGSOFT International Workshop on Automating TEST Case Design, Selection, and Evaluation

Self Cite

View full text Add to dashboard Cite

Context: Albeit different approaches exist for automated GUI testing of hybrid mobile applications, the practice appears to be not so commonly adopted by developers. A possible reason for such a low diffusion can be the fragility of the techniques, i.e. the frequent need for maintaining test cases when the GUI of the app is changed.Goal: In this paper, we perform an assessment of the maintenance needed by test cases for a hybrid mobile app, and the related fragility causes.Methods: We evaluated a small test suite with a Layout-based testing tool (Appium) and a Visual one (EyeAutomate) and observed the changes needed by tests during the co-evolution with the GUI of the app.Results: We found that 20% Layout-based test methods and 30% Visual test methods had to be modified at least once, and that each release induced fragilities in 3-4% of the test methods.Conclusion: Fragility of GUI tests can induce relevant maintenance efforts in test suites of large applications. Several principal causes for fragilities have been identified for the tested hybrid application, and guidelines for developers are deduced from them. CCS CONCEPTS• Software and its engineering → Software verification and validation; Software testing and debugging;

show abstract

Scripted GUI testing of Android open-source apps: evolution of test code and fragility causes

Cited by 21 publications

References 41 publications

Automated Test Selection for Android Apps Based on APK and Activity Classification

Automated Test Selection for Android Apps Based on APK and Activity Classification

Characterizing the transition to Kotlin of Android apps: a study on F-Droid, Play Store, and GitHub

Fragility of layout-based and visual GUI test scripts: an assessment study on a hybrid mobile application

Contact Info

Product

Resources

About