A multi-objective approach to search-based test data generation

Lakhotia, Kiran; Harman, Mark; McMinn, Phil

doi:10.1145/1276958.1277175

Cited by 108 publications

(65 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Recently, Daka et al [14] used a post-processing technique to optimize readability by mutating generated tests leveraging a domain-specific model of unit test readability based on human judgement. Other non coverage-based criteria exploited in literature for test case generation include execution time [17,37], memory consumption [25], test size [19,31,35], and ability to reveal faults [37].…”

Section: Introductionmentioning

confidence: 99%

Automatic test case generation: what if test code quality matters?

Palomba

Panichella

Zaidman

et al. 2016

Proceedings of the 25th International Symposium on Software Testing and Analysis

View full text Add to dashboard Cite

Test case generation tools that optimize code coverage have been extensively investigated. Recently, researchers have suggested to add other non-coverage criteria, such as memory consumption or readability, to increase the practical usefulness of generated tests. In this paper, we observe that test code quality metrics, and test cohesion and coupling in particular, are valuable candidates as additional criteria. Indeed, tests with low cohesion and/or high coupling have been shown to have a negative impact on future maintenance activities. In an exploratory investigation we show that most generated tests are indeed affected by poor test code quality. For this reason, we incorporate cohesion and coupling metrics into the main loop of search-based algorithm for test case generation. Through an empirical study we show that our approach is not only able to generate tests that are more cohesive and less coupled, but can (i) increase branch coverage up to 10% when enough time is given to the search and (ii) result in statistically shorter tests.

show abstract

Section: Introductionmentioning

confidence: 99%

Automatic test case generation: what if test code quality matters?

Palomba

Panichella

Zaidman

et al. 2016

Proceedings of the 25th International Symposium on Software Testing and Analysis

View full text Add to dashboard Cite

show abstract

“…One popular example is the multi-objective optimization problem, where the challenge is to identify 'the best' solution to a problem that involves numerous factors. This is commonly solved with the help of evolutionary algorithms, and is an example of a MADA problem that has featured extensively in software testing research [15].…”

Section: Evidential Reasoningmentioning

confidence: 99%

Using evidential reasoning to make qualified predictions of software quality

Walkinshaw

2013

Proceedings of the 9th International Conference on Predictive Models in Software Engineering

View full text Add to dashboard Cite

Software quality is commonly characterised in a top-down manner. High-level notions such as quality are decomposed into hierarchies of sub-factors, ranging from abstract notions such as maintainability and reliability to lower-level notions such as test coverage or team-size. Assessments of abstract factors are derived from relevant sources of information about their respective lower-level sub-factors, by surveying sources such as metrics data and inspection reports. This can be difficult because (1) evidence might not be available, (2) interpretations of the data with respect to certain quality factors may be subject to doubt and intuition, and (3) there is no straightforward means of blending hierarchies of heterogeneous data into a single coherent and quantitative prediction of quality. This paper shows how Evidential Reasoning (ER) -a mathematical technique for reasoning about uncertainty and evidence -can address this problem. It enables the quality assessment to proceed in a bottomup manner, by the provision of low-level assessments that make any uncertainty explicit, and automatically propagating these up to higher-level 'belief-functions' that accurately summarise the developer's opinion and make explicit any doubt or ignorance.

show abstract

“…Though our work is the first to use a multi-objective approach to service-based testing. Lakhotia et al [13], Oster and Saglietti [18] and Pinto and Vergilio [19] already proposed multi-objective test data generation. These approaches focus on structural coverage as the main objective and use additional objectives such as execution time, memory consumption and size of test set.…”

Section: Test Data Generation and Optimisationmentioning

confidence: 99%

Optimised Realistic Test Input Generation Using Web Services

Bozkurt

Harman

2012

Search Based Software Engineering

Self Cite

View full text Add to dashboard Cite

Abstract. We introduce a multi-objective formulation of service-oriented testing, focusing on the balance between service price and reliability. We experimented with NSGA2 for this problem, investigating the effect on performance and quality of composition size, topology and the number of services discovered. For topologies small enough for exhaustive search we found that NSGA2 finds a pareto front very near (the fronts are a Euclidean distance of ∼ 0.00024 price-reliability points apart) the true pareto front. Regarding performance, we find that composition size has the strongest effect, with smaller topologies consuming more machine time; a curious effect we believe is due to the influence of crowding distance. Regarding result quality, our results reveal that size and topology have more effect on the front found than the number of service choices discovered. As expected the cost-reliability relationship (logarithmic, linear, exponential) is replicated in the front discovered when correlation is high, but as the price-reliability correlation decreases, we find fewer solutions on the front and the front becomes less smooth.

show abstract

A multi-objective approach to search-based test data generation

Cited by 108 publications

References 19 publications

Automatic test case generation: what if test code quality matters?

Automatic test case generation: what if test code quality matters?

Using evidential reasoning to make qualified predictions of software quality

Optimised Realistic Test Input Generation Using Web Services

Contact Info

Product

Resources

About