The Influence of Code Coverage Metrics on Automated Testing Efficiency in Android

Dashevskyi, Stanislav; Gadyatskaya, Olga; Pilgun, Aleksandr; Zhauniarovich, Yury

doi:10.1145/3243734.3278524

Cited by 7 publications

(11 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, activity coverage is a prerequisite condition to reveal crashes that might happen during the interaction with the app's UI. The more coverage a tool explores, the higher the chances a potential crash can be found [44]. In this study, the number of inputs generated by a tool within a time limit was measured.…”

Section: Case Study Criteriamentioning

confidence: 99%

“…Method Coverage (MC): MC is the ratio of the number of methods called during execution of the AUT to the total number of methods in the source code of the app. By improving the method coverage, it is envisaged that most of the app's functionalities are explored and tested [8,38,39,44].…”

Section: Case Study Criteriamentioning

confidence: 99%

“…In the literature, different studies have used the number of unique crashes detected as the primary evaluation criteria. The higher the crash number detected (in comparison to other testing tools), the better the tool performance in detecting app crashes [44]. Figure 6 shows the distribution of crashes in each testing tool.…”

Section: Rq2 How Is the Performance Of The Test Input Generation Toomentioning

confidence: 99%

See 2 more Smart Citations

An Empirical Analysis of Test Input Generation Tools for Android Apps through a Sequence of Events

et al. 2020

View full text Add to dashboard Cite

Graphical User Interface (GUI) testing of Android apps has gained considerable interest from the industries and research community due to its excellent capability to verify the operational requirements of GUI components. To date, most of the existing GUI testing tools for Android apps are capable of generating test inputs by using different approaches and improve the Android apps’ code coverage and fault detection performance. Many previous studies have evaluated the code coverage and crash detection performances of GUI testing tools in the literature. However, very few studies have investigated the effectiveness of the test input generation tools, especially in the events sequence length of the overall test coverage and crash detection. The event sequence length generally shows the number of steps required by the test input generation tools to detect a crash. It is critical to highlight its effectiveness due to its significant effects on time, testing effort, and computational cost. Thus, this study evaluated the effectiveness of six test input generation tools for Android apps that support the system events generation on 50 Android apps. The generation tools were evaluated and compared based on the activity coverage, method coverage, and capability in detecting crashes. Through a critical analysis of the results, this study identifies the diversity and similarity of test input generation tools for Android apps to provide a clear picture of the current state of the art. The results revealed that a long events sequence performed better than a shorter events sequence. However, a long events sequence led to a minor positive effect on the coverage and crash detection. Moreover, the study showed that the tools achieved less than 40% of the method coverage and 67% of the activity coverage.

show abstract

Section: Case Study Criteriamentioning

confidence: 99%

Section: Case Study Criteriamentioning

confidence: 99%

Section: Rq2 How Is the Performance Of The Test Input Generation Toomentioning

confidence: 99%

See 1 more Smart Citation

An Empirical Analysis of Test Input Generation Tools for Android Apps through a Sequence of Events

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Techniques for collecting coverage information from the execution of software systems remain a topic of current investigation [2].…”

Section: Scalable Minimally-disruptive Fine Grained Coveragementioning

confidence: 99%

Some challenges for software testing research (invited talk paper)

Alshahwan

Ciancone

Harman

et al. 2019

Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis

View full text Add to dashboard Cite

“…Nowadays, there are mainly several criteria [4]- [8] for judging the adequacy of APP testing. First, it is activity coverage.…”

Section: Introductionmentioning

confidence: 99%

Theoretical Analysis and Empirical Evaluation of Coverage Indictors for Closed Source APP Testing

Huang

Hui

2019

IEEE Access

View full text Add to dashboard Cite

The number of mobile applications has increased geometrically nowadays, but how to ensure their quality and conduct adequate and effective testing is still a challenge for developers. On the one hand, the number of mobile apps is increasing, and the update speed is faster and faster. Many small and mediumsized companies can hardly test the app adequately before each release. On the other hand, mobile apps play more and more important roles in people's life, such as financial payment. For the sake of company security and user privacy, most companies will encrypt key codes in their APP. Even third-party testers cannot get source code, which also leads to many researchers cannot carry out further research and effective testing for these widely used mainstream APPs. Code coverage is an important indicator to guide software testing, which plays a crucial role in ensuring the quality of testing. However, it is an urgent problem to find accurate coverage indictors to evaluate these tests. And when testing those existing widely used mainstream closed-source apps, we find that the existing coarse-grained coverage metrics like method coverage is bad coverage indictors for app testing that can exaggerate or minimize the actual coverage rate, which cannot obtain satisfactory results for the evaluation of test effects. To find a more reliable coverage indictor, this paper demonstrates the correctness of instruction coverage indictor and the inaccurate of method coverage in evaluating the test of closed-source APP from the perspective of probability and statistics. Then we shows how inaccurate the method coverage can be through an empirical evaluation on datasets of closed source APPs and open source APPs respectively. It is further verified that instruction coverage is a more effective evaluation indictor than methods coverage or activity coverage in the test of closed source APP for the first time. INDEX TERMS Mobile testing, automated testing, coverage criteria, instruction coverage.

show abstract

The Influence of Code Coverage Metrics on Automated Testing Efficiency in Android

Cited by 7 publications

References 3 publications

An Empirical Analysis of Test Input Generation Tools for Android Apps through a Sequence of Events

An Empirical Analysis of Test Input Generation Tools for Android Apps through a Sequence of Events

Some challenges for software testing research (invited talk paper)

Theoretical Analysis and Empirical Evaluation of Coverage Indictors for Closed Source APP Testing

Contact Info

Product

Resources

About