PBlaman: performance blame analysis based on Palladio contracts

Abstract

SUMMARYIn performance-driven software engineering, the performance of a system is evaluated through models before the system is assembled. After assembly, the performance is then validated using performance tests. When a component-based system fails certain performance requirements during the tests, it is important to find out whether individual components yield performance errors or whether the composition of components is faulty. This task is called performance blame analysis. Existing performance blame analysis approaches and also alternative error analysis approaches are restricted, because they either do not employ expected values, use expected values from regression testing, or use static developer-set limits. In contrast, this paper describes the new performance blame analysis approach PBlaman that builds upon our previous work and that employs the context-portable performance contracts of Palladio. PBlaman decides what components to blame by comparing the observed response time data series of each single component operation in a failed test case to the operation's expected response time data series derived from the contracts. System architects are then assisted by a visual presentation of the obtained analysis results. We exemplify the benefits of PBlaman in two case studies, each of which representing applications that follow a particular architectural style.

A Short Survey on Using Software Error Localization for Service Compositions

“…So far, to the best of our knowledge, automated, tool-based approaches for localizing faults in models of service compositions do not exist, at least when it comes to functional correctness, i.e., the adherence of the model to functional requirements. With respect to performance analysis of systems, feasible approaches to localize components that negatively impact the overall performance of the system, have been devised in the area of performance blame analysis [18,13].…”

Performance Prediction of Service-Oriented Architecture - A survey

“…The message is generated in XML but it is based on specifications defined in WSDL, step 6 the response generated by the provider is also in tag based XML format. Based on ISO 9126 performance metrics are response time, throughput, and resource utilization [12]. Therefore, accurate measuring of SOA application plays an important role to business success.…”

A special issue from the international conference on performance engineering 2013

“…The extended papers submitted went through a new round of reviews, and this led to the selection of the six contributions that appear here. We would like to thank the authors of all the extended papers for their diligence in preparing their manuscripts for consideration and the members of the special issue Program Committee for their many insightful and helpful suggestions.The contributions include a new analytical model for estimating the survivability of a smart power grid [1]; a method for identifying ('blaming') software components and/or component compositions that result in performance errors in a component-based application [2]; a new performance model for Hadoop's MapReduce framework that is based on modeling separately the generic and application-specific phases of the framework [3]; a measurement-based study of the performance of web servers running on multi-core systems that can be used to understand the scaling behavior of a web server and to help find server configurations that optimize performance for a given workload [4]; a method for dynamically switching between different time-series-based forecasting methods for an evolving workload trace whose behavior is controlled solely by user-defined forecasting objectives [5]; a resampling technique that constructs a workload model from a supercomputer job trace that can then be used to conduct a simulation-based analysis of the performance of a parallel job scheduler [6].This rich diversity of research contributions to performance engineering reflects the evolution of the field and the roots of ICPE, which evolved from a performance evaluation workshop sponsored by the Standard Performance Evaluation Corporation (SPEC) and from the Workshop on Software and Performance (WOSP). While the SPEC workshop focused on benchmarking and encompassed many areas of software and hardware evaluation, the main focus of WOSP was the development of methodologies for software performance modeling, prediction, and engineering.…”

“…The contributions include a new analytical model for estimating the survivability of a smart power grid ; a method for identifying (‘blaming’) software components and/or component compositions that result in performance errors in a component‐based application ; a new performance model for Hadoop's MapReduce framework that is based on modeling separately the generic and application‐specific phases of the framework ; a measurement‐based study of the performance of web servers running on multi‐core systems that can be used to understand the scaling behavior of a web server and to help find server configurations that optimize performance for a given workload ; a method for dynamically switching between different time‐series‐based forecasting methods for an evolving workload trace whose behavior is controlled solely by user‐defined forecasting objectives ; a resampling technique that constructs a workload model from a supercomputer job trace that can then be used to conduct a simulation‐based analysis of the performance of a parallel job scheduler .…”