In search-based software engineering we often use popular heuristics with default configurations, which typically lead to suboptimal results, or we perform experiments to identify configurations on a trialand-error basis, which may lead to better results for a specific problem. To obtain better results while avoiding trial-and-error experiments, a fitness landscape analysis is helpful in understanding the search problem, and making an informed decision about the heuristics. In this paper, we investigate the search problem of test suite generation for mobile applications (apps) using Sapienz whose heuristic is a default NSGA-II. We analyze the fitness landscape of Sapienz with respect to genotypic diversity and use the gained insights to adapt the heuristic of Sapienz. These adaptations result in Sapienz div that aims for preserving the diversity of test suites during the search. To evaluate Sapienz div , we perform a head-to-head comparison with Sapienz on 76 open-source apps.
Context: In search-based software engineering we often use popular heuristics with default configurations, which typically lead to suboptimal results, or we perform experiments to identify configurations on a trial-and-error basis, which may lead to better results for a specific problem. We consider the problem of generating test suites for mobile applications (apps) and rely on Sapienz, a state-of-the-art approach to this problem that uses a popular heuristic (NSGA-II) with a default configuration. Objective: We want to achieve better results in generating test suites with Sapienz while avoiding trial-and-error experiments to identify a more suitable configuration of Sapienz. Method: We conducted a fitness landscape analysis of Sapienz to analytically understand the search problem, which allowed us to make informed decisions about the heuristic and configuration of Sapienz when developing Sapienz div. We comprehensively evaluated Sapienz div in a headto-head comparison with Sapienz on 34 apps. Results: Analyzing the fitness landscape of Sapienz, we observed a lack of diversity of the evolved test suites and a stagnation of the search after 25 generations. Sapienz div realizes mechanisms that preserve the diversity of the test suites being evolved. The evaluation showed that Sapienz div achieves better or at least similar test results than Sapienz concerning coverage and the number of revealed faults. However, Sapienz div typically produces longer test sequences and requires more execution time than Sapienz. Conclusions: The understanding of the search problem obtained by the fitness landscape analysis helped us to find a more suitable configuration of Sapienz without trial-and-error experiments. By promoting diversity of test suites during the search, improved or at least similar test results in terms of faults and coverage can be achieved.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.