The performance of enterprise software systems has a direct impact on the success of business. Recent studies have shown that software performance affects customer satisfaction as well as operational costs of software. Hence, software performance constitutes an essential competitive and differentiating factor for software vendors and operators. In industrial practice, it is still a challenging task to detect software performance problems before they are faced by end users. Diagnostics of performance problems requires deep expertise in performance engineering and still entails a high manual effort. As a consequence, performance evaluations are postponed to the last minute of the development process, or even are completely omitted. Instead of proactively avoiding performance problems, problems are fixed in a reactive manner when they first emerge in operations. Since reactive, operation-time resolution of performance problems is very expensive and has a negative impact on the reputation of software vendors, performance problems need to be diagnosed and resolved in the process of software development. Existing approaches addressing performance problem diagnostics either assume the existence of a performance model, are limited to problem detection without analyzing root causes, or are applied as reactive approaches during the operations phase and, thus, cannot be effective applied during development for performance problem diagnostics.In this thesis, we introduce an automatic, experiment-based approach for performance problem diagnostics in enterprise software systems. We describe a method to derive a taxonomy on recurrent types of performance problems and introduce a systematic experimentation concept. Using the taxonomy as a search tree, the proposed approach systematically searches for root causes of detected performance problems by executing series of systematic performance experiments. Based on the measurement data from experiments, detection heuristics decide on the presence of performance problems in the target system. Furthermore, we develop a domain-specific description language to specify the information required for automatic performance problem diagnostics. Finally, we create and evaluate a representative set of detection heuristics. We validate our approach by means of five studies including end-to-end case studies, a controlled experiment and an empirical study. The results of the validation show that our approach is applicable to a wide range of contexts and is able to fully automatically and accurately detect performance problems in medium-size and large-scale applications. External users of the provided approach evaluated it as a useful support for diagnostics of performance problems and exposed their willingness to use the approach for their own software development projects. Explicitly designed for automatic, development-time testing, our approach can be incorporated into continuous integration. In this way, our approach allows regular, automatic diagnostics of performance problems involving minima...

No abstract

Instrumentation and monitoring plays an important role in measurement-based performance analysis of software systems. However, in practice the performance overhead of extensive instrumentation is not negligible. Experiment-based performance analysis overcomes this problem through a series of experiments on selectively instrumented code, but requires additional manual effort to adjust required instrumentation and hence introduces additional costs. Automating the experiments and selective instrumentation can massively reduce the costs of performance analysis. Such automation, however, requires the capability of dynamically adapting instrumentation instructions. In this paper, we address this issue by introducing AIM, a novel instrumentation and monitoring approach for automated software performance analysis. We apply AIM to automate derivation of resource demands for an architectural performance model, showing that adaptable instrumentation leads to more accurate measurements compared to existing monitoring approaches.

Cloud environments reduce data center operating costs through resource sharing and economies of scale. Infrastructure-as-a-Service is one example that leverages virtualization to share infrastructure resources. However, virtualization is often insufficient to provide Software-as-a-Service applications due to the need to replicate the operating system, middleware and application components for each customer. To overcome this problem, multi-tenancy has emerged as an architectural style that allows to share a single Web application instance among multiple independent customers, thereby significantly improving the efficiency of Software-as-a-Service offerings. A number of platforms are available today that support the development and hosting of multi-tenant applications by encapsulating multi-tenancy specific functionality. Although a lack of performance guarantees is one of the major obstacles to the adoption of cloud computing, in general, and multi-tenant applications, in particular, these kinds of applications and platforms have so far not been in the focus of the performance and benchmarking community. In this paper, we present an extended version of an existing and widely accepted application benchmark adding support for multi-tenant platform features. The benchmark is focused on evaluating the maximum throughput and the amount of tenants that can be served by a platform. We present a case study comparing virtualization and multi-tenancy. The results demonstrate the practical usability of the proposed benchmark in evaluating multi-tenant platforms and gives insights that help to decide for one sharing approach.

Performance problems such as high response times in software applications have a significant effect on the customer's satisfaction. In enterprise applications, performance problems are frequently manifested in inefficient or unnecessary communication patterns between software components originating from poor architectural design or implementation. Due to high manual effort, thorough performance analysis is often neglected, in practice. In order to overcome this problem, automated engineering approaches are required for the detection of performance problems. In this paper, we introduce several heuristics for measurement-based detection of well-known performance anti-patterns in inter-component communications. The detection heuristics comprise load and instrumentation descriptions for performance tests as well as corresponding detection rules. We integrate these heuristics with Dynamic Spotter, a framework for automatic detection of performance problems. We evaluate our heuristics on four evaluation scenarios based on an e-commerce benchmark (TPC-W) where the heuristics detect the expected communication performance anti-patterns and pinpoint their root causes.