Capturing performance assumptions using stochastic performance logic

Abstract

Compared to functional unit testing, automated performance testing is difficult, partially because correctness criteria are more difficult to express for performance than for functionality. Where existing approaches rely on absolute bounds of the execution time, we aim to express assertions on code performance in relative, hardware-independent terms. To this end, we introduce Stochastic Performance Logic (SPL), which allows making statements about relative method performance. Since SPL interpretation is based on statistical tests applied to performance measurements, it allows (for a special class of formulas) calculating the minimum probability at which a particular SPL formula holds. We prove basic properties of the logic and present an algorithm for SAT-solver-guided evaluation of SPL formulas, which allows optimizing the number of performance measurements that need to be made. Finally, we propose integration of SPL formulas with Java code using higher-level performance annotations, for performance testing and documentation purposes.

Constructing performance model of JMS middleware platform

“…Besides simple software documentation purposes, this use can also benefit software performance testing [5,15]. In this context, the models can be provided with the same inputs as in the early stages of the software development process, with one important addition -the models can be automatically calibrated against the actual performance in selected benchmarks.…”

Towards Performance-Aware Engineering of Autonomic Component Ensembles

“…We capture performance goals by Stochastic Performance Logic (SPL) [9], which is a many-sorted first-order logic with well-defined semantics. SPL regards performance as a random variable with probability distribution dependent on a given workload.…”

“…Testing of isolated execution requires a developer (tester) to provide a sample workload. In traditional performance unit testing as described in [9], the tester needs to prepare a workload generator that creates the parameters for the method under test. When testing DEECo components, the tester has to provide artificial knowledge upon which the component can operate.…”

Adaptive deployment in ad-hoc systems using emergent component ensembles

“…In our previous work [8] we proposed to use Stochastic Performance Logic (SPL) [9] to express rules for adaptation in component systems based on real and predicted performance of individual components. The rules controlling the adaptation are similar to the decision logic of the Planner that compares deployment alternatives for Ab.…”