Comparing the use of bayesian networks and neural networks in response time modeling for service-oriented systems

Abstract

The new paradigm of service-oriented computing facilitates easy construction of dynamic, complex distributed systems. Recent research has shown that machine learning methods can be a promising way to autonomously and accurately derive models to assist autonomic management software or humans in understanding system behaviors and making informed decisions. However, the efficacy of different machine learning techniques in describing various system behaviors and meeting distinct application needs has not been systematically understood. Such an understanding can prove crucial in management infrastructure design and implementation for service-oriented systems.This paper is an initial step to bridge the gap and specifically contrasts the applications of Bayesian networks (BN) and neural networks (NN) in modeling the response time of service-oriented systems. Relatively simple BN and NN models are designed and implemented as a base of the comparison study. As far as model performance is concerned, a wide range of simulations show that BNs offer better accuracy, are less sensitive to small data set size and are therefore more suited for environments that change rapidly and need frequent response time model reconstructions; whereas NNs can achieve faster model evaluation time and support management routines that demand intensive response time predictions. From a non-performance perspective, it is analytically concluded that BNs can be more easily understood by human and support multi-direction evaluation, while NNs provide more flexible response time representation.

Statistical Approaches to Predicting and Diagnosing Performance Problems in Component-Based Distributed Systems: An Experimental Evaluation

“…However, as observed by Kohavi in [7], no algorithm can outperform all others in every case and, thus, more comparative studies are important. The relevance of such studies is also recognized by Zhang et al in [8], in which the authors highlight the need for reviewing the applicability and suitability of different techniques for the autonomic management field (see Section IV). Beyond the forecasting dimension, few efforts have applied statistical learning techniques for diagnostic purposes.…”

“…There have been few works concentrating on evaluating the fit and applicability of different statistical learning techniques to deal with performance problems. A comparative study of Bayesian network and neural networks for modeling response time in service-oriented systems is presented by Zhang et al in [8]. However, only Bayesian network and neural networks are considered in the study and the effect of different instantiations and parametrizations is not explored in the work.…”

Translation of application-level terms to resource-level attributes across the Cloud stack layers

“…An interesting work, comparing Bayesian Networks (BNs) with ANNs is presented in [24]. According to this conclusion, ANNs have better model response times but worse model creation times.…”

Modeling the behavior of large scale reasoning systems using clustering and regression

“…For example [5] apply neural networks and Bayesian networks for modeling the response time of service-oriented computing facilities in the construction of dynamic, complex distributed systems. A radial basis function neural network has been applied by [6] for modeling the performance of a parallel I/O system with experiments on IBM SP.…”