Optimised Realistic Test Input Generation Using Web Services

Abstract

Abstract. We introduce a multi-objective formulation of service-oriented testing, focusing on the balance between service price and reliability. We experimented with NSGA2 for this problem, investigating the effect on performance and quality of composition size, topology and the number of services discovered. For topologies small enough for exhaustive search we found that NSGA2 finds a pareto front very near (the fronts are a Euclidean distance of ∼ 0.00024 price-reliability points apart) the true pareto front. Regarding performance, we find that composition size has the strongest effect, with smaller topologies consuming more machine time; a curious effect we believe is due to the influence of crowding distance. Regarding result quality, our results reveal that size and topology have more effect on the front found than the number of service choices discovered. As expected the cost-reliability relationship (logarithmic, linear, exponential) is replicated in the front discovered when correlation is high, but as the price-reliability correlation decreases, we find fewer solutions on the front and the front becomes less smooth.

Cost-aware pareto optimal test suite minimisation for service-centric systems

“…Reliability of a test input is calculated by ATAM as the combined reliability of the data sources used in generation of this input. The reliability calculation and data source selection in ATAM are discussed in an earlier work [3].…”

“…ATAM also calculates the reliability of each generated realistic input based on the services used in the input generation process. Details of our formulation for the input reliability score is available in an earlier paper [3].…”

“…The reliability of each input is calculated using the formulation we described in our previous work [2,3]. Previously, we introduced the concept of service-centric test data generation addressing the issues of automation in realistic test data generation [2].…”

“…According to literature, one of the limitations of Servicecentric System(s) Testing (ScST) is the cost associated with invoking services [3,4,6]. The cost of invoking services during testing is a major problem at composition level.…”