Methods for Live Testing of Cloud Services

Abstract

Service providers use cloud to reduce the cost of their services and speed up their time to market. There is a huge gap between the environment where the services are developed and first tested, and the environment where they operate. Live testing is defined as testing a service in its production environment without causing any intolerable disruption to its usage. It is considered as one of the solutions to overcome the impact of the difference between the development environment and the production environment on the reliability of the test results. Test interferences are a major challenge for live testing as they may lead to the system mishandling the production traffic. Existing solutions to alleviate the risk of interferences have limited applicability. In this paper we propose a set of test methods, applicable in different situations, for live testing to improve the reliability of test results without causing any intolerable disturbance to the cloud services. To reduce the complexity of these test methods we define the concept of boundary environments and a set of coverage criteria to aim at during live testing.

Test Plan Generation for Live Testing of Cloud Services

“…In a previous work Jebbar et al (2020b) we proposed a set of test methods that can be used to perform live testing of cloud services. These test methods are applicable in an environment that supports 1) snapshotting and cloning of components; and 2) service relocation as means of state transfer between components.…”

“…The sum of these assigned numbers of occurrences is called the mixture width. Jebbar et al (2020b) describes a set of coverage criteria for mixture of width one (also known as paths), as well as coverage criteria for mixtures of arbitrary width.…”

“…In this work we did not propose test isolation techniques, but we propose a test isolation strategy (Algorithm 2) which helps select the right test isolation technique for each situation. The set of test isolation techniques on which we based our solution is the one desribed in Jebbar et al (2020b).…”

“…The generated test plan is described using the UML Testing Profile (UTP) OMG (2020), and it is based 2 on the architectural and modeling framework proposed in Jebbar et al (2020a). The test methods used to run the test cases in production have been proposed in Jebbar et al (2020b). The main goal of our test plan generation method is to reduce the time needed to execute the generated test plan without inducing any unacceptable disturbance.…”

“…We provide some background in Section 3. In Section 4 we discuss the modeling using UTP of the test methods introduced in Jebbar et al (2020b). In Section 5 we introduce and discuss our test plan generation method.…”