Web Services Wind Tunnel: On Performance Testing Large-Scale Stateful Web Services

Barros, Márcio de Oliveira; Shiau, J.; Shang, Chen; Gidewall, K.; Shi, Hui; Forsmann, J.

doi:10.1109/dsn.2007.102

Cited by 21 publications

(18 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The work from Santiago et al [35] proposes the use of state charts for the automated test case generation (our goal is to use a similar approach to generate the workload). Another approach, based on de Barros et al [36], is to generate the workload using the characterization of real load patterns through the application of Markov chains.…”

Section: Fig 3 Example Of a Wsdl Filementioning

confidence: 99%

A robustness testing approach for SOAP Web services

Laranjeiro

Vieira

Madeira

2012

J Internet Serv Appl

View full text Add to dashboard Cite

The use of Web services in enterprise applications is quickly increasing. In a Web services environment, providers supply a set of services for consumers. However, although Web services are being used in business-critical environments, there are no practical means to test or compare their robustness to invalid and malicious inputs. In fact, client applications are typically developed with the assumption that the services being used are robust, which is not always the case. Robustness failures in such environments are particularly dangerous, as they may originate vulnerabilities that can be maliciously exploited, with severe consequences for the systems under attack. This paper addresses the problem of robustness testing in Web services environments. The proposed approach is based on a set of robustness tests (including both malicious and non-malicious invalid call parameters) that is used to discover programming and design errors. This approach, useful for both service providers and consumers, is demonstrated by two sets of experiments, showing, respectively, the use of Web services Robustness testing from the consumer and the provider points of view. The experiments comprise the robustness testing of 1,204 Web service operations publicly available in the Internet and of 29 home-implemented services, including two different implementations of the Web services specified by the standard TPC-App performance benchmark. Results show that many Web services are deployed with critical robustness N. Laranjeiro · M. Vieira (B) · H. Madeira DEI/CISUC, University of Coimbra, 3030-290 Coimbra, Portugal e-mail: mvieira@dei.uc.pt N. Laranjeiro e-mail: cnl@dei.uc.pt H. Madeira e-mail: henrique@dei.uc.pt problems and that robustness testing is an effective approach to improve services quality.

show abstract

Section: Fig 3 Example Of a Wsdl Filementioning

confidence: 99%

A robustness testing approach for SOAP Web services

Laranjeiro

Vieira

Madeira

2012

J Internet Serv Appl

View full text Add to dashboard Cite

show abstract

“…This includes, for instance, tests for performance and Quality of Service (QoS) [16], [7], robustness [17], reliability [8], [18], message schema conformance [19], but also techniques for testing composed services [9], [10] as well as generic and customizable testing tools [6]. In spite of their importance, these solutions only support engineers in checking the service providers of a SOA.…”

Section: A Related Research On Soa Testingmentioning

confidence: 99%

“…In general, the lack of proper testing support has been regarded as one of the main problems of SOA [5]. Looking at currently available solutions, it becomes evident that the majority aims only at testing of single Web services [6], [7], [8] and composite ones [9], [10] which, however, only covers the service provider part of SOA. For testing systems which operate in service-based environments themselves, the engineer is facing the problem of setting up realistic test scenarios which cover the system's whole functionality.…”

Section: Introductionmentioning

confidence: 99%

Script-Based Generation of Dynamic Testbeds for SOA

Juszczyk

Dustdar

2011

Socially Enhanced Services Computing

View full text Add to dashboard Cite

Abstract-This paper addresses one of the major problems of SOA software development: the lack of support for testing complex service-oriented systems. The research community has developed various means for checking individual Web services but has not come up with satisfactory solutions for testing systems that operate in service-based environments and, therefore, need realistic testbeds for evaluating their quality. We regard this as an unnecessary burden for SOA engineers. As a proposed solution for this issue, we present the Genesis2 testbed generator framework. Genesis2 supports engineers in modeling testbeds and programming their behavior. Out of these models it generates running instances of Web services, clients, registries, and other entities in order to emulate realistic SOA environments. By generating real testbeds, our approach assists engineers in performing runtime tests of their systems and particular focus has been put on the framework's extensibility to allow the emulation of arbitrarily complex environments. Furthermore, by exploiting the advantages of the Groovy language, Genesis2 provides an intuitive yet powerful scripting interface for testbed control.

show abstract

“…This problem needs to be addressed early in the development phase. As a consequence, much effort has been put into the development of methods and tools for automated testing and detection of error-prone components in SOAs [18,26,30,34,36,40]. However, those solutions mainly aim at analyzing only individual Web services by performing various client-oriented checks.…”

Section: Introductionmentioning

confidence: 99%

GENESIS - A Framework for Automatic Generation and Steering of Testbeds of ComplexWeb Services

Juszczyk

Truong

Dustdar

2008

13th IEEE International Conference on Engineering of Complex Computer Systems (Iceccs 2008)

View full text Add to dashboard Cite

Nowadays, the importance of Web services is steadily increasing in domains where interoperability is of paramount importance. This trend is especially observable in complex computer systems which consist of a large number of interacting distributed components, often implemented using Web services. Large-scale systems and high complexity usually result in higher error-proneness in the development process. This should be addressed as early as possible during the development of complex service-oriented systems, ideally before they are actually deployed on a distributed infrastructure. In this paper we present GENESISa software framework for solving this problem. Our framework allows automatic generation and steering of testbeds of complex Web services, thereby empowering developers to specify functional and non-functional properties of Web services, to generate and deploy Web service instances on remote hosting environments, to enhance the functionality of the framework with plug-ins, and to control the behavior of the testbed during runtime.

show abstract

Web Services Wind Tunnel: On Performance Testing Large-Scale Stateful Web Services

Cited by 21 publications

References 4 publications

A robustness testing approach for SOAP Web services

A robustness testing approach for SOAP Web services

Script-Based Generation of Dynamic Testbeds for SOA

GENESIS - A Framework for Automatic Generation and Steering of Testbeds of ComplexWeb Services

Contact Info

Product

Resources

About