Analysing the fitness landscape of search-based software testing problems

Aleti, Aldeida; Moser, Irene; Grunske, Lars

doi:10.1007/s10515-016-0197-7

Cited by 40 publications

(28 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our experiments indicate a dominance of plateau branches in object‐oriented classes, which leads to a difficult search landscape. Similar findings have been reported by recent attempts to classify the search landscape . One proposed way to improve problematic search landscapes in search‐based testing is to apply testability transformation .…”

Section: Impact Of the Findingssupporting

confidence: 83%

Random or evolutionary search for object‐oriented test suite generation?

Shamshiri

Rojas

Gazzola

et al. 2018

Software Testing Verif & Rel

View full text Add to dashboard Cite

Summary An important aim in software testing is constructing a test suite with high structural code coverage, that is, ensuring that most if not all of the code under test have been executed by the test cases comprising the test suite. Several search‐based techniques have proved successful at automatically generating tests that achieve high coverage. However, despite the well‐established arguments behind using evolutionary search algorithms (eg, genetic algorithms) in preference to random search, it remains an open question whether the benefits can actually be observed in practice when generating unit test suites for object‐oriented classes. In this paper, we report an empirical study on the effects of using evolutionary algorithms (including a genetic algorithm and chemical reaction optimization) to generate test suites, compared with generating test suites incrementally with random search. We apply the EVOSUITE unit test suite generator to 1000 classes randomly selected from the SF110 corpus of open‐source projects. Surprisingly, the results show that the difference is much smaller than one might expect: While evolutionary search covers more branches of the type where standard fitness functions provide guidance, we observed that, in practice, the vast majority of branches do not provide any guidance to the search. These results suggest that, although evolutionary algorithms are more effective at covering complex branches, a random search may suffice to achieve high coverage of most object‐oriented classes.

show abstract

Section: Impact Of the Findingssupporting

confidence: 83%

Random or evolutionary search for object‐oriented test suite generation?

Shamshiri

Rojas

Gazzola

et al. 2018

Software Testing Verif & Rel

View full text Add to dashboard Cite

show abstract

“…• For data with multiple versions, we test on the latest version and train on a combination of all the rest. • If FFtrees perform worse than any other learner by more than a "small effect" (defined using Equation 5), then that result is highlighted in red (see the synapse d2h results of Figure 3). For each column, the size of a "small effect" is listed at top.…”

Section: Fig 3: Defect Prediction Results For Fftree Vs Untuned Learmentioning

confidence: 99%

How to “DODGE” Complex Software Analytics

Agrawal

Wei

Chen

et al. 2021

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Machine learning techniques applied to software engineering tasks can be improved by hyperparameter optimization, i.e., automatic tools that find good settings for a learner's control parameters. We show that such hyperparameter optimization can be unnecessarily slow, particularly when the optimizers waste time exploring "redundant tunings", i.e., pairs of tunings which lead to indistinguishable results. By ignoring redundant tunings, DODGE(E), a tuning tool, runs orders of magnitude faster, while also generating learners with more accurate predictions than seen in prior state-of-the-art approaches.

show abstract

“…Furthermore, we intend to enhance the EvoChecker synthesis capabilities by supporting other evolutionary and natured-inspired optimisation algorithms like evolutionary strategies, particle swarm optimisation and ant-colony optimisation (Coello et al 2006). Finally, adapting techniques that analyse the fitness landscape of the induced search space (Aleti et al 2017) is another possible extension for EvoChecker.…”

Section: Discussionmentioning

confidence: 99%

Synthesis of probabilistic models for quality-of-service software engineering

Gerasimou

Călinescu

Tamburrelli³

2018

Autom Softw Eng

View full text Add to dashboard Cite

An increasingly used method for the engineering of software systems with strict quality-of-service (QoS) requirements involves the synthesis and verification of probabilistic models for many alternative architectures and instantiations of system parameters. Using manual trial-and-error or simple heuristics for this task often produces suboptimal models, while the exhaustive synthesis of all possible models is typically intractable. The EvoChecker search-based software engineering approach presented in our paper addresses these limitations by employing evolutionary algorithms to automate the model synthesis process and to significantly improve its outcome. EvoChecker can be used to synthesise the Pareto-optimal set of probabilistic models associated with the QoS requirements of a system under design, and to support the selection of a suitable system architecture and configuration. EvoChecker can also be used at runtime, to drive the efficient reconfiguration of a self-adaptive software system. We evaluate EvoChecker on several variants of three systems from different application domains, and show its effectiveness and applicability. Keywords Search-based software engineering • Probabilistic model checking • Evolutionary algorithms • QoS requirements B Simos Gerasimou

show abstract

Analysing the fitness landscape of search-based software testing problems

Cited by 40 publications

References 39 publications

Random or evolutionary search for object‐oriented test suite generation?

Random or evolutionary search for object‐oriented test suite generation?

How to “DODGE” Complex Software Analytics

Synthesis of probabilistic models for quality-of-service software engineering

Contact Info

Product

Resources

About