Towards a structural load testing tool

Yang, Cheer-Sun; Pollock, Lori

doi:10.1145/226295.226318

Cited by 14 publications

(8 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, analyzing source code and system behavior are the main general approaches used in the literature for designing stress test conditions based on fault-inducing workload. Deriving loads using dataflow analysis and symbolic execution are examples of techniques for designing fault-inducing load based on source code analysis to detect functional (like memory leaks) and performance (exceeded response time) problems [28,58]. Using linear programs and genetic algorithms are the techniques used for designing loads based on system behavior analysis to detect the performance problems [29,30,31] Our proposed framework for autonomous stress testing uses reinforcement learning to learn how to generate stress test conditions for finding performance breaking point of software systems.…”

Section: Related Workmentioning

confidence: 99%

“…Testing also raises issues of automated and efficient generation of test conditions (test cases). Using source code analysis [27,28], linear programs and genetic algorithms based on system models like performance models [29,30,31] and UML models [32,33,34,35,36], and using use case-based design approaches [37,38] are some common approaches in performance testing for generating test conditions to detect the functional and non-functional problems. While relying on source code or other artifacts might raise issues of limitations upon unavailability of the source code and other artifacts.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An autonomous performance testing framework using self-adaptive fuzzy reinforcement learning

et al. 2021

View full text Add to dashboard Cite

Test automation brings the potential to reduce costs and human effort, but several aspects of software testing remain challenging to automate. One such example is automated performance testing to find performance breaking points. Current approaches to tackle automated generation of performance test cases mainly involve using source code or system model analysis or use-case-based techniques. However, source code and system models might not always be available at testing time. On the other hand, if the optimal performance testing policy for the intended objective in a testing process instead could be learned by the testing system, then test automation without advanced performance models could be possible. Furthermore, the learned policy could later be reused for similar software systems under test, thus leading to higher test efficiency. We propose SaFReL, a self-adaptive fuzzy reinforcement learning-based performance testing framework. SaFReL learns the optimal policy to generate performance test cases through an initial learning phase, then reuses it during a transfer learning phase, while keeping the learning running and updating the policy in the long term. Through multiple experiments in a simulated performance testing setup, we demonstrate that our approach generates the target performance test cases for different programs more efficiently than a typical testing process and performs adaptively without access to source code and performance models.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

An autonomous performance testing framework using self-adaptive fuzzy reinforcement learning

et al. 2021

View full text Add to dashboard Cite

show abstract

“…In addition, test inputs for load testing can be generated simultaneously while the real SUT executes (online) or independently (offline). In the offline approach, test loads are designed from the source code using static analysis techniques, such as data flow analysis and symbolic execution , using the operational profile (workload characterization) or using design models annotated with statistics derived from the operational profile and past data, such as Unified Modeling Language (UML) use‐case diagrams , Markov chains , and probabilistic time automata . In the online approach, there is feedback from the real system to the test generation process to refine test loads.…”

Section: Literature Reviewmentioning

confidence: 99%

An end‐user‐centric test generation methodology for performance evaluation of mobile networked applications

Al‐tekreeti

Abdrabou

Naik

2019

Software Testing Verif & Rel

View full text Add to dashboard Cite

We propose a model-based test generation methodology to evaluate the impact of the interaction of the wireless network and application configurations on the performance of mobile networked applications. We consider waiting time delay to model wireless network quality. We classify mobile applications into two groups. Group I represents applications where end-user experience is mainly affected by waiting time delay during service consumption, while group II represents applications where end-user experience is affected by waiting time delay before service consumption. Test generation is formulated as an inversion problem. However, for group I applications, solving the inversion problem is expensive. Therefore, we utilize metamorphic testing to mitigate the cost of test oracles. We formulate metamorphic test generation as maximization of the distance between seed and follow-up test cases. Two test coverage criteria are proposed: user experience and user-experience-and-input interaction. Network models are developed for a mobile device that has network access through a WiFi hot spot and uses either transmission control protocol or user datagram protocol. Two mobile applications are used to demonstrate the methodology: multimedia streaming and web browsing. Application of the methodology when user actions are taken into consideration is also addressed. The effectiveness of the methodology is evaluated using two metrics: the incurred time cost and redundancy in the generated test suite. The obtained results show the advantage of casting test generation as an inversion problem, compared with random testing. For apps with intensive performance models, combining metamorphic testing with the methodology has tremendously reduced the cost of test oracles. 15 of 38 state y. Therefore, the SUT desired behaviour introduces the following two constraints (Procedure 3.S1), where B, M , and L are defined as an integer number of multimedia frames:M Ä B and L Ä M 1:(4) 4.1.2. The wireless network model. The end user is streaming via a WiFi hot spot and using the UDP. We assume all the fluctuations in the wireless link and the queuing effects of the different routers along the path from the server to the client device manifest as a time delay. That is, packet loss is negligible. A reusable network model for the UDP scenario was developed in our previous work [15]. We have developed two analytical expressions for the mean and variance of packet interarrival time delay. The probability distribution was matched with the hyper-Erlang distribution. The network impact is captured by three operating parameters (NOPs): the data rate (D), the mean rate of frame arrivals into the access point ( ) per user, and the number of end users (N ) connected to the AP. The data rate according to the IEEE 802.11 a/g standard can be in one of the following values: 6, 9, 12, 18, 24, 32, 48, or 54 Mbps. It mainly relates to the quality of the wireless connection between the AP and the end user. Regarding the number of users N , the network model is validated...

show abstract

“…Similar trends appear in load testing techniques and processes in general as they use other sources of information (e.g., user profiles [1], adaptive resource models [2]) to decide how to induce a given load, but still operating from a black box perspective. One interesting exception proposed by Yang et al [27]. Conceptually, the approach aims to assign load sensitivity indexes to software modules based on their potential to allocate memory, and use that information to drive load testing.…”

Section: Related Workmentioning

confidence: 99%

Automatic generation of load tests

Zhang

Elbaum

Dwyer

2011

2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011)

View full text Add to dashboard Cite

Load tests aim to validate whether system performance is acceptable under peak conditions. Existing test generation techniques induce load by increasing the size or rate of the input. Ignoring the particular input values, however, may lead to test suites that grossly mischaracterize a system's performance. To address this limitation we introduce a mixed symbolic execution based approach that is unique in how it 1) favors program paths associated with a performance measure of interest, 2) operates in an iterative-deepening beam-search fashion to discard paths that are unlikely to lead to high-load tests, and 3) generates a test suite of a given size and level of diversity. An assessment of the approach shows it generates test suites that induce program response times and memory consumption several times worse than the compared alternatives, it scales to large and complex inputs, and it exposes a diversity of resource consuming program behavior.

show abstract

Towards a structural load testing tool

Cited by 14 publications

References 9 publications

An autonomous performance testing framework using self-adaptive fuzzy reinforcement learning

An autonomous performance testing framework using self-adaptive fuzzy reinforcement learning

An end‐user‐centric test generation methodology for performance evaluation of mobile networked applications

Automatic generation of load tests

Contact Info

Product

Resources

About