Code Coverage Measurement for Android Dynamic Analysis Tools

Huang, Chun-Ying; Chiu, Ching-Hsiang; Lin, Chih-Hung; Tzeng, Han-Wei

doi:10.1109/mobserv.2015.38

Cited by 14 publications

(23 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is the ratio of triggered instruction in the Java instruction code of the app to the total number of instructions. Huang et al [68] first proposed the concept of instruction coverage, which is used in many studies as an indicator to evaluate test efficiency [24,44,54,64]. It is a more accurate and valid test coverage criterion that reflects the adequacy of testing results for closed-source apps [25].…”

Section: Case Study Criteriamentioning

confidence: 99%

DroidbotX: Test Case Generation Tool for Android Applications Using Q-Learning

2021

View full text Add to dashboard Cite

Android applications provide benefits to mobile phone users by offering operative functionalities and interactive user interfaces. However, application crashes give users an unsatisfactory experience, and negatively impact the application’s overall rating. Android application crashes can be avoided through intensive and extensive testing. In the related literature, the graphical user interface (GUI) test generation tools focus on generating tests and exploring application functions using different approaches. Such tools must choose not only which user interface element to interact with, but also which type of action to be performed, in order to increase the percentage of code coverage and to detect faults with a limited time budget. However, a common limitation in the tools is the low code coverage because of their inability to find the right combination of actions that can drive the application into new and important states. A Q-Learning-based test coverage approach developed in DroidbotX was proposed to generate GUI test cases for Android applications to maximize instruction coverage, method coverage, and activity coverage. The overall performance of the proposed solution was compared to five state-of-the-art test generation tools on 30 Android applications. The DroidbotX test coverage approach achieved 51.5% accuracy for instruction coverage, 57% for method coverage, and 86.5% for activity coverage. It triggered 18 crashes within the time limit and shortest event sequence length compared to the other tools. The results demonstrated that the adaptation of Q-Learning with upper confidence bound (UCB) exploration outperforms other existing state-of-the-art solutions.

show abstract

Section: Case Study Criteriamentioning

confidence: 99%

DroidbotX: Test Case Generation Tool for Android Applications Using Q-Learning

2021

View full text Add to dashboard Cite

show abstract

“…In many industrial widely used APPs, line coverage is often unavailable because many testers, as third-party testers, are unable to access the source code of these closed-source APPs, which hinders the development of testing tools, because the performance of these tools on complex closed-source APPs cannot be judged without accurate criteria. Reference [35] first proposed the concept of instruction coverage, which used the Smali [36] code instructions by decompiling the APK installation package, which released by APP developers, as the statistical indictor of test coverage. Unfortunately, there has been no work to prove the correctness of this criteria,…”

Section: ) Line Coverage and Instruction Coveragementioning

confidence: 99%

Theoretical Analysis and Empirical Evaluation of Coverage Indictors for Closed Source APP Testing

Huang

Hui

2019

IEEE Access

View full text Add to dashboard Cite

The number of mobile applications has increased geometrically nowadays, but how to ensure their quality and conduct adequate and effective testing is still a challenge for developers. On the one hand, the number of mobile apps is increasing, and the update speed is faster and faster. Many small and mediumsized companies can hardly test the app adequately before each release. On the other hand, mobile apps play more and more important roles in people's life, such as financial payment. For the sake of company security and user privacy, most companies will encrypt key codes in their APP. Even third-party testers cannot get source code, which also leads to many researchers cannot carry out further research and effective testing for these widely used mainstream APPs. Code coverage is an important indicator to guide software testing, which plays a crucial role in ensuring the quality of testing. However, it is an urgent problem to find accurate coverage indictors to evaluate these tests. And when testing those existing widely used mainstream closed-source apps, we find that the existing coarse-grained coverage metrics like method coverage is bad coverage indictors for app testing that can exaggerate or minimize the actual coverage rate, which cannot obtain satisfactory results for the evaluation of test effects. To find a more reliable coverage indictor, this paper demonstrates the correctness of instruction coverage indictor and the inaccurate of method coverage in evaluating the test of closed-source APP from the perspective of probability and statistics. Then we shows how inaccurate the method coverage can be through an empirical evaluation on datasets of closed source APPs and open source APPs respectively. It is further verified that instruction coverage is a more effective evaluation indictor than methods coverage or activity coverage in the test of closed source APP for the first time. INDEX TERMS Mobile testing, automated testing, coverage criteria, instruction coverage.

show abstract

“…The final outcome from this study was orientated through building of a body of knowledge which will be considered an important asset to any researcher or developer in this field. A study about the process of obtaining code coverage for Android dynamic analysis tools was realized by [8] which disclose their approach as a set of connected processes between decompiling an apk file, inserting instrumentation code, recompiling, repackaging and testing the final product modified. They report a low rate of successfully repackaging Android apk file, situated at the value of only 36%, but the code coverage results were similar to those prefabricated with EMMA tool.…”

Section: Literature Reviewmentioning

confidence: 99%

Android App Code Coverage

Mihail-Vaduva¹

2019

Proceedings of the 18th International Conference on INFORMATICS in ECONOMY Education, Research and Business Technologies

View full text Add to dashboard Cite

Activities related with software quality evaluation for mobile devices are nowadays confronted with a major outpouring of trends related to outstanding mobile evaluation possibilities. In this paper we use statement code coverage (CC), a software quality metric, in order to discover technical conditions needed to accomplish when we will desire to check and prove that the same pattern of manually testing an Android application was really used, through a beta testing process, by an end user. For this mission we are using the regression analysis. Scientific articles were studied concerning different ways to understand and evaluate calculation of statement code coverage for mobile software application and the relationship between existing methods features and the choice that a developer could have against one or another method to evaluate this quality software metric. Our starting point related with this article, that makes a difference, is motivated by the fact that we use a tool that operates in a black box style, without the need to know the Java Android application's source code. Our quest is orientated in the direction of integrate our practical experiment into an educational laboratory process to a better learning of programming and testing software applications for mobile devices.

show abstract

Code Coverage Measurement for Android Dynamic Analysis Tools

Cited by 14 publications

References 5 publications

DroidbotX: Test Case Generation Tool for Android Applications Using Q-Learning

DroidbotX: Test Case Generation Tool for Android Applications Using Q-Learning

Theoretical Analysis and Empirical Evaluation of Coverage Indictors for Closed Source APP Testing

Android App Code Coverage

Contact Info

Product

Resources

About