Deriving performance-relevant infrastructure properties through model-based experiments with Ginpex

Hauck, Michael; Kuperberg, Michael; Huber, Nikolaus; Reussner, Ralf

doi:10.1007/s10270-013-0335-7

Cited by 5 publications

(5 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this way, experiments can be specified with different load constellations and progression. With respect to systematic experimentation, the difference between the work in (Hauck et al, 2011) and our SSE is the same as it was the case for the work of D. J. Westermann, 2014.…”

Section: Experimentation and Models In Measurement-based Performance mentioning

confidence: 79%

“…For the derivation of performance-relevant properties of infrastructures Hauck et al introduce the GINPEX approach (Hauck et al, 2011). Using a custom load driver, GINPEX applies multiple experiments with different load profiles to analyze the performance-relevant parameters of the infrastructure under test.…”

Section: Experimentation and Models In Measurement-based Performance mentioning

confidence: 99%

“…Regarding the experiment specification language of Hauck et al, there are similarities to P 2 D 2 M with respect to the intention of the individual model parts, including specification of the measurement environment, metrics and experiment series. However, as the focus in (Hauck et al, 2011) is on performance analysis of infrastructure properties, their model is tailored for that specific purpose and, thus, aims at modelling other aspects than the P 2 D 2 M.…”

Section: Experimentation and Models In Measurement-based Performance mentioning

confidence: 99%

See 2 more Smart Citations

Performance problem diagnostics by systematic experimentation

Wert

2013

Proceedings of the 18th International Doctoral Symposium on Components and Architecture

View full text Add to dashboard Cite

The performance of enterprise software systems has a direct impact on the success of business. Recent studies have shown that software performance affects customer satisfaction as well as operational costs of software. Hence, software performance constitutes an essential competitive and differentiating factor for software vendors and operators. In industrial practice, it is still a challenging task to detect software performance problems before they are faced by end users. Diagnostics of performance problems requires deep expertise in performance engineering and still entails a high manual effort. As a consequence, performance evaluations are postponed to the last minute of the development process, or even are completely omitted. Instead of proactively avoiding performance problems, problems are fixed in a reactive manner when they first emerge in operations. Since reactive, operation-time resolution of performance problems is very expensive and has a negative impact on the reputation of software vendors, performance problems need to be diagnosed and resolved in the process of software development. Existing approaches addressing performance problem diagnostics either assume the existence of a performance model, are limited to problem detection without analyzing root causes, or are applied as reactive approaches during the operations phase and, thus, cannot be effective applied during development for performance problem diagnostics.In this thesis, we introduce an automatic, experiment-based approach for performance problem diagnostics in enterprise software systems. We describe a method to derive a taxonomy on recurrent types of performance problems and introduce a systematic experimentation concept. Using the taxonomy as a search tree, the proposed approach systematically searches for root causes of detected performance problems by executing series of systematic performance experiments. Based on the measurement data from experiments, detection heuristics decide on the presence of performance problems in the target system. Furthermore, we develop a domain-specific description language to specify the information required for automatic performance problem diagnostics. Finally, we create and evaluate a representative set of detection heuristics. We validate our approach by means of five studies including end-to-end case studies, a controlled experiment and an empirical study. The results of the validation show that our approach is applicable to a wide range of contexts and is able to fully automatically and accurately detect performance problems in medium-size and large-scale applications. External users of the provided approach evaluated it as a useful support for diagnostics of performance problems and exposed their willingness to use the approach for their own software development projects. Explicitly designed for automatic, development-time testing, our approach can be incorporated into continuous integration. In this way, our approach allows regular, automatic diagnostics of performance problems involving minima...

show abstract

Section: Experimentation and Models In Measurement-based Performance mentioning

confidence: 79%

Section: Experimentation and Models In Measurement-based Performance mentioning

confidence: 99%

Section: Experimentation and Models In Measurement-based Performance mentioning

confidence: 99%

See 1 more Smart Citation

Performance problem diagnostics by systematic experimentation

Wert

2013

Proceedings of the 18th International Doctoral Symposium on Components and Architecture

View full text Add to dashboard Cite

show abstract

“…Both methods use monitored measurements on the The second group of related work deals with benchmarking and performance analysis of virtualized environments not specifically targeted at storage systems. Hauck et al [13] propose a goal-oriented measurement approach to determine performance-relevant infrastructure properties. They examine OS scheduler properties and CPU virtualization overhead.…”

Section: Related Workmentioning

confidence: 99%

Predictive performance modeling of virtualized storage systems using optimized statistical regression techniques

Noorshams

Bruhn

Kounev

et al. 2013

Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering

Self Cite

View full text Add to dashboard Cite

Modern virtualized environments are key for reducing the operating costs of data centers. By enabling the sharing of physical resources, virtualization promises increased resource efficiency with decreased administration costs. With the increasing popularity of I/O-intensive applications, however, the virtualized storage used in such environments can quickly become a bottleneck and lead to performance and scalability issues. Performance modeling and evaluation techniques applied prior to system deployment help to avoid such issues. In current practice, however, virtualized storage and its performance-influencing factors are often neglected or treated as a black-box. In this paper, we present a measurement-based performance prediction approach for virtualized storage systems based on optimized statistical regression techniques. We first propose a general heuristic search algorithm to optimize the parameters of regression techniques. Then, we apply our optimization approach and create performance models using four regression techniques. Finally, we present an in-depth evaluation of our approach in a real-world representative environment based on IBM System z and IBM DS8700 server hardware. Using our optimized techniques, we effectively create performance models with less than 7% prediction error in the most typical scenario. Furthermore, our optimization approach reduces the prediction error by up to 74%.

show abstract

“…The micro-benchmarks were implemented with the Ginpex experiment framework [52]. The CPU load of the micro-benchmarks consists of the calculation of Fibonacci numbers, the number of iterations is calibrated by Ginpex before an experiment run to match the desired resource demand.…”

Section: Dataset D2: Micro-benchmarksmentioning

confidence: 99%