Model-based performance testing in the cloud using the mbpet tool

Abbors, Fredrik; Ahmad, Tanwir; Truşcan, Dragoş; Porres, Iván

doi:10.1145/2479871.2479937

Cited by 16 publications

(10 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Timed Automata Compared to the Markov Chain and the Stochastic Form-oriented Models, Probability Timed Automata is an abstraction which provides support for user action modeling as well as timing delays [108], [109], [110]. Similar to the Markov chain model, a Probabilistic Timed Automata contains a set of states and transition probabilities between states.…”

Section: ) Testing Loads Derived Using Probabilisticmentioning

confidence: 99%

A Survey on Load Testing of Large-Scale Software Systems

Jiang

Hassan²

2015

IIEEE Trans. Software Eng.

139

View full text Add to dashboard Cite

Many large-scale software systems must service thousands or millions of concurrent requests. These systems must be load tested to ensure that they can function correctly under load (i.e., the rate of the incoming requests). In this paper, we survey the state of load testing research and practice. We compare and contrast current techniques that are used in the three phases of a load test: (1) designing a proper load, (2) executing a load test, and (3) analyzing the results of a load test. This survey will be useful for load testing practitioners and software engineering researchers with interest in the load testing of large-scale software systems.

show abstract

Section: ) Testing Loads Derived Using Probabilisticmentioning

confidence: 99%

A Survey on Load Testing of Large-Scale Software Systems

Jiang

Hassan²

2015

IIEEE Trans. Software Eng.

139

View full text Add to dashboard Cite

show abstract

“…However, efficiency was specified in (13) requirements, from which we conclude that efficiency is difficult to document and quantify. We found few examples of quantified efficiency requirements: (1) The external server data store containing RLCS status for use by external systems shall be updated once per minute, 9 and (2) The system must accomplish 90% for transactions in less than 1 second. 10 The examples show that it is possible to quantify efficiency.…”

Section: Discussionmentioning

confidence: 99%

“…In the SMS, there were threats related to the data extraction methods. (1) We may have missed some papers because two databases used by Dias Neto et al [19] we did not have access to. To keep this to a minimum we made sure that we use the SCOPUS database, which includes publications from different technical publishers.…”

Section: Threats To Validitymentioning

confidence: 99%

An approach for performance requirements verification and test environments generation

2022

View full text Add to dashboard Cite

Model-based testing (MBT) is a method that supports the design and execution of test cases by models that specify the intended behaviors of a system under test. While systematic literature reviews on MBT in general exist, the state of the art on modeling and testing performance requirements has seen much less attention. Therefore, we conducted a systematic mapping study on model-based performance testing. Then, we studied natural language software requirements specifications in order to understand which and how performance requirements are typically specified. Since none of the identified MBT techniques supported a major benefit of modeling, namely identifying faults in requirements specifications, we developed the Performance Requirements verificatiOn and Test EnvironmentS generaTion approach (PRO-TEST). Finally, we evaluated PRO-TEST on 149 requirements specifications. We found and analyzed 57 primary studies from the systematic mapping study and extracted 50 performance requirements models. However, those models don’t achieve the goals of MBT, which are validating requirements, ensuring their testability, and generating the minimum required test cases. We analyzed 77 Software Requirements Specification (SRS) documents, extracted 149 performance requirements from those SRS, and illustrate that with PRO-TEST we can model performance requirements, find issues in those requirements and detect missing ones. We detected three not-quantifiable requirements, 43 not-quantified requirements, and 180 underspecified parameters in the 149 modeled performance requirements. Furthermore, we generated 96 test environments from those models. By modeling performance requirements with PRO-TEST, we can identify issues in the requirements related to their ambiguity, measurability, and completeness. Additionally, it allows to generate parameters for test environments.

show abstract

“…In addition, test inputs for load testing can be generated simultaneously while the real SUT executes (online) or independently (offline). In the offline approach, test loads are designed from the source code using static analysis techniques, such as data flow analysis and symbolic execution , using the operational profile (workload characterization) or using design models annotated with statistics derived from the operational profile and past data, such as Unified Modeling Language (UML) use‐case diagrams , Markov chains , and probabilistic time automata . In the online approach, there is feedback from the real system to the test generation process to refine test loads.…”

Section: Literature Reviewmentioning

confidence: 99%

An end‐user‐centric test generation methodology for performance evaluation of mobile networked applications

Al‐tekreeti

Abdrabou

Naik

2019

Software Testing Verif & Rel

View full text Add to dashboard Cite

We propose a model-based test generation methodology to evaluate the impact of the interaction of the wireless network and application configurations on the performance of mobile networked applications. We consider waiting time delay to model wireless network quality. We classify mobile applications into two groups. Group I represents applications where end-user experience is mainly affected by waiting time delay during service consumption, while group II represents applications where end-user experience is affected by waiting time delay before service consumption. Test generation is formulated as an inversion problem. However, for group I applications, solving the inversion problem is expensive. Therefore, we utilize metamorphic testing to mitigate the cost of test oracles. We formulate metamorphic test generation as maximization of the distance between seed and follow-up test cases. Two test coverage criteria are proposed: user experience and user-experience-and-input interaction. Network models are developed for a mobile device that has network access through a WiFi hot spot and uses either transmission control protocol or user datagram protocol. Two mobile applications are used to demonstrate the methodology: multimedia streaming and web browsing. Application of the methodology when user actions are taken into consideration is also addressed. The effectiveness of the methodology is evaluated using two metrics: the incurred time cost and redundancy in the generated test suite. The obtained results show the advantage of casting test generation as an inversion problem, compared with random testing. For apps with intensive performance models, combining metamorphic testing with the methodology has tremendously reduced the cost of test oracles. 15 of 38 state y. Therefore, the SUT desired behaviour introduces the following two constraints (Procedure 3.S1), where B, M , and L are defined as an integer number of multimedia frames:M Ä B and L Ä M 1:(4) 4.1.2. The wireless network model. The end user is streaming via a WiFi hot spot and using the UDP. We assume all the fluctuations in the wireless link and the queuing effects of the different routers along the path from the server to the client device manifest as a time delay. That is, packet loss is negligible. A reusable network model for the UDP scenario was developed in our previous work [15]. We have developed two analytical expressions for the mean and variance of packet interarrival time delay. The probability distribution was matched with the hyper-Erlang distribution. The network impact is captured by three operating parameters (NOPs): the data rate (D), the mean rate of frame arrivals into the access point ( ) per user, and the number of end users (N ) connected to the AP. The data rate according to the IEEE 802.11 a/g standard can be in one of the following values: 6, 9, 12, 18, 24, 32, 48, or 54 Mbps. It mainly relates to the quality of the wireless connection between the AP and the end user. Regarding the number of users N , the network model is validated...

show abstract

Model-based performance testing in the cloud using the mbpet tool

Cited by 16 publications

References 4 publications

A Survey on Load Testing of Large-Scale Software Systems

A Survey on Load Testing of Large-Scale Software Systems

An approach for performance requirements verification and test environments generation

An end‐user‐centric test generation methodology for performance evaluation of mobile networked applications

Contact Info

Product

Resources

About