Automatic Extraction of Session-Based Workload Specifications for Architecture-Level Performance Models

Vögele, Christian; Hoorn, André van; Krcmar, Helmut

doi:10.1145/2693182.2693183

Cited by 4 publications

(1 citation statement)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…DevOps principles may help to derive more representative and effective load scripts. Applying approaches for automatic workload derivation during operation (Vögele et al, 2015;van Hoorn et al, 2008;van Hoorn et al, 2014) to derive up-to-date, representative load scripts for APPD provides for more accurate diagnostics results. Furthermore, high-level monitoring during operations may provide initial hypotheses that may more efficiently guide performance problem diagnostics in the testing phase.…”

Section: Long-term Research Directionsmentioning

confidence: 99%

Performance problem diagnostics by systematic experimentation

Wert

2013

Proceedings of the 18th International Doctoral Symposium on Components and Architecture

View full text Add to dashboard Cite

The performance of enterprise software systems has a direct impact on the success of business. Recent studies have shown that software performance affects customer satisfaction as well as operational costs of software. Hence, software performance constitutes an essential competitive and differentiating factor for software vendors and operators. In industrial practice, it is still a challenging task to detect software performance problems before they are faced by end users. Diagnostics of performance problems requires deep expertise in performance engineering and still entails a high manual effort. As a consequence, performance evaluations are postponed to the last minute of the development process, or even are completely omitted. Instead of proactively avoiding performance problems, problems are fixed in a reactive manner when they first emerge in operations. Since reactive, operation-time resolution of performance problems is very expensive and has a negative impact on the reputation of software vendors, performance problems need to be diagnosed and resolved in the process of software development. Existing approaches addressing performance problem diagnostics either assume the existence of a performance model, are limited to problem detection without analyzing root causes, or are applied as reactive approaches during the operations phase and, thus, cannot be effective applied during development for performance problem diagnostics.In this thesis, we introduce an automatic, experiment-based approach for performance problem diagnostics in enterprise software systems. We describe a method to derive a taxonomy on recurrent types of performance problems and introduce a systematic experimentation concept. Using the taxonomy as a search tree, the proposed approach systematically searches for root causes of detected performance problems by executing series of systematic performance experiments. Based on the measurement data from experiments, detection heuristics decide on the presence of performance problems in the target system. Furthermore, we develop a domain-specific description language to specify the information required for automatic performance problem diagnostics. Finally, we create and evaluate a representative set of detection heuristics. We validate our approach by means of five studies including end-to-end case studies, a controlled experiment and an empirical study. The results of the validation show that our approach is applicable to a wide range of contexts and is able to fully automatically and accurately detect performance problems in medium-size and large-scale applications. External users of the provided approach evaluated it as a useful support for diagnostics of performance problems and exposed their willingness to use the approach for their own software development projects. Explicitly designed for automatic, development-time testing, our approach can be incorporated into continuous integration. In this way, our approach allows regular, automatic diagnostics of performance problems involving minima...

show abstract

Section: Long-term Research Directionsmentioning

confidence: 99%

Performance problem diagnostics by systematic experimentation

Wert

2013

Proceedings of the 18th International Doctoral Symposium on Components and Architecture

View full text Add to dashboard Cite

show abstract

WESSBAS: extraction of probabilistic workload specifications for load testing and performance prediction—a model-driven approach for session-based application systems

Vögele

Hoorn

Schulz³

et al. 2016

Softw Syst Model

Self Cite

View full text Add to dashboard Cite

The specification of workloads is required in order to evaluate performance characteristics of application systems using load testing and model-based performance prediction. Defining workload specifications that represent the real workload as accurately as possible is one of the biggest challenges in both areas. To overcome this challenge, this paper presents an approach that aims to automate the extraction and transformation of workload specifications for load testing and model-based performance prediction of session-based application systems. The approach (WESSBAS) comprises three main components. First, a system- and tool-agnostic domain-specific language (DSL) allows the layered modeling of workload specifications of session-based systems. Second, instances of this DSL are automatically extracted from recorded session logs of production systems. Third, these instances are transformed into executable workload specifications of load generation tools and model-based performance evaluation tools. We present transformations to the common load testing tool Apache JMeter and to the Palladio Component Model. Our approach is evaluated using the industry-standard benchmark SPECjEnterprise2010 and the World Cup 1998 access logs. Workload-specific characteristics (e.g., session lengths and arrival rates) and performance characteristics (e.g., response times and CPU utilizations) show that the extracted workloads match the measured workloads with high accuracy.

show abstract

Lodestone: A Streaming Approach to Behavior Modeling and Load Testing

Parrott

Carver

2020

2020 3rd International Conference on Data Intelligence and Security (ICDIS)

View full text Add to dashboard Cite

Automatic Extraction of Session-Based Workload Specifications for Architecture-Level Performance Models

Cited by 4 publications

References 17 publications

Performance problem diagnostics by systematic experimentation

Performance problem diagnostics by systematic experimentation

WESSBAS: extraction of probabilistic workload specifications for load testing and performance prediction—a model-driven approach for session-based application systems

Lodestone: A Streaming Approach to Behavior Modeling and Load Testing

Contact Info

Product

Resources

About