Ensembles of Models for Automated Diagnosis of System Performance Problems

Abstract

Violations of service level objectives (SLO) in Internet services are urgent conditions requiring immediate attention. Previously we showed [1] that Tree-Augmented Bayesian Networks or TAN models are effective at identifying which low-level system properties were correlated to high-level SLO violations (the metric attribution problem) under stable workloads. In this paper we extend our approach to adapt to changing workloads and external disturbances by maintaining an ensemble of probabilistic models, adding new models when existing ones do not accurately capture current system behavior. Using realistic workloads on an implemented prototype system, we show that the ensemble of TAN models captures the performance behavior of the system accurately under changing workloads and conditions. We fuse diagnoses from the ensemble of models to identify likely causes of the performance problem, with results comparable to those produced by an oracle that continuously changes the model based on advance knowledge of the workload. The cost of inducing new models and managing the ensembles is negligible, making our approach both immediately practical and theoretically appealing.

Sensor Selection for IT Infrastructure Monitoring

“…Evaluating large amounts of measured metrics by statistical methods and methods from artificial intelligence can be effectively utilized to improve enterprise systems" dependability by allowing fault detection [4,5] and the forecasting of the system"s behavior [6,7]. Unfortunately, the dimensional problem of such approaches has not been sufficiently addressed by the community.…”

Adaptive Learning of Metric Correlations for Temperature-Aware Database Provisioning

“…Cohen et al [8] and Zhang et al [28] use a similar approach for estimating performance models by means of statistical learning. They use tree-augmented Bayesian networks (TAN) to discover correlations between system metrics and service level objectives (SLO).…”

CloudPD: Problem determination and diagnosis in shared dynamic clouds

“…Note that these online models are trained only with the historical data portions free of anomalies. We leverage existing techniques [1], [9], [18] for the methodology used in this phase (see Section IV-B).…”

“…However, this technique only works for positive correlations. Zhang et al [18] leverage an ensemble of models to address variations in an underlying model (performance or correlation-based) that may change with time due to fluctuation in workload intensity or mix, software or hardware updates. Similarly, Cherkasova et al [15] identify an application change using two different models for a given time period, leading to higher accuracy.…”