Model driven benchmark generation for web services

Abstract

Web services solutions are being increasingly adopted in enterprise systems. However, ensuring the quality of service of Web services applications remains a costly and complicated performance engineering task. Some of the new challenges include limited controls over consumers of a service, unforeseeable operational scenarios and vastly different XML payloads. These challenges make existing manual performance analysis and benchmarking methods difficult to use effectively. This paper describes an approach for generating customized benchmark suites for Web services applications from a software architecture description following a Model Driven Architecture (MDA) approach. We have provided a performance-tailored version of the UML 2.0 Testing Profile so architects can model a flexible and reusable load testing architecture, including test data, in a standards compatible way. We extended our MDABench [27] tool to provide a Web service performance testing "cartridge" associated with the tailored testing profile. A load testing suite and automatic performance measurement infrastructure are generated using the new cartridge. Best practices in Web service testing are embodied in the cartridge and inherited by the generated code. This greatly reduces the effort needed for Web service performance benchmarking while being fully MDA compatible. We illustrate the approach using a case study on the Apache Axis platform.

Designing service-based applications in the presence of non-functional properties: A mapping study

“…Middleware [198], [57], [77], [110], [210], [36], [67], [131], [90], [76], [185] [13], [88], [29], [39], [145], [130], [165], [111], [166], [34], [172], [217], [187], [66], [37], [52], [94], [2], [10], [152], [82], [195], [3], [5], [12], [26], [31], [45], [92], [101], [211], [127], [137], [138], [140], [170], [173], [69], [164].…”

“…Model-Driven [47], [130], [176], [217], [16], [4], [129], [18], [195], [21], [63], [87], [51], [156], [9].…”

“…Efficiency [196], [43], [8], [167], [34], [217], [42], [60], [189], [86], [179], [1], [51], [204], [93], [140], [178], [175]. Maintainability [1], [51], [93].…”

Validating Model-Driven Performance Predictions on Random Software Systems

“…Besides validating a particular methodology, the question of prediction precision is tackled from many other angles, including workload characterization [29,16], or generation of either the executable system [31,13,33,6] or the performance model [15,19]. What these approaches have in common is that they still require Self-archived copy.…”

Model-Based Generation of Testbeds for Web Services

“…Closer to our approach are some QoS testbeds. Zhu and coauthors [26] develop an interesting model-driven approach to the generation of benchmarks reproducing clients' workload for evaluating the QoS provided by a Web Service. Grundy and coauthors [15] propose a performance test-bed generator which has much in common with what we propose here, in that a collection of service stubs is generated from a compositional model, and clients simulating a defined workload are also automatically developed.…”