Achievements, Open Problems and Challenges for Search Based Software Testing

Abstract

Abstract-Search Based Software Testing (SBST) formulates testing as an optimisation problem, which can be attacked using computational search techniques from the field of Search Based Software Engineering (SBSE). We present an analysis of the SBST research agenda 1 , focusing on the open problems and challenges of testing non-functional properties, in particular a topic we call 'Search Based Energy Testing' (SBET), Multi-objective SBST and SBST for Test Strategy Identification. We conclude with a vision of FIFIVERIFY tools, which would automatically find faults, fix them and verify the fixes. We explain why we think such FIFIVERIFY tools constitute an exciting challenge for the SBSE community that already could be within its reach.

Mobile app and app store analysis, testing and optimisation

“…Recent advances in automated test input generation [11] make it possible to automatically generate inputs that cover, for example, white box structure or subtle faults that may be present [9,23]. Such advances in testing have helped make possible a new approach to software improvement that has come to be known as 'Genetic Improvement' [13,18,26].…”

Automated Transplantation of Call Graph and Layout Features into Kate

“…Clearly one can never do enough regression testing, but these results provide release some confidence. In future work we plan to use automated search based software testing [5] to further improve autotransplantation regression testing.…”

“…Like much previous work on genetic programming [12], our approach relies critically on the availability of high quality test suites. We do not directly address this issue in the present paper, but believe that existing achievements in Search Based [5] and other [4] test data generation techniques will help us to ensure that this reliance is reasonable and practical.…”

Testing Software Using Swarm Intelligence: A Bee Colony Optimization Approach

“…Our third contribution lies in the objective function used. Previous work in search based testing focused on a single objective function, which is mainly decision coverage [10,11]. Our fitness function is multi objective, where the focus is on achieving condition and decision coverage, while multi-condition coverage is considered as a bonus objective.…”