Predictive models for proactive network management: application to a production Web server

Shen, Dan; Hellerstein, Joseph L.

doi:10.1109/noms.2000.830432

Cited by 33 publications

(23 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then, from (2), using the full basis, we can write (11) where are the coefficients associated with the complete basis decomposition, is the matrix of all the basis functions, and its inverse has an analysis filter bank interpretation. In other words, denoting the th row of by , (11) can be written as (12) which is nothing but the output obtained on passing through a filter bank consisting of finite-impulse-response (FIR) filters, with being the impulse response of the th filter.…”

Section: A Determination Of the Fixed Parametersmentioning

confidence: 99%

“…In this paper, we follow the hierarchical approach proposed in [11] and [12] in which the time-series prediction is decomposed into two steps: first a prediction of the long-term component, which primarily captures the nonstationarity of the data, is performed, and then the residual short-term process, which captures both the long-term prediction error and the short-term component of the time series, is processed. As demonstrated by the results, the two-scale decomposition captures the underlying statistics of the data fairly well.…”

Section: Introductionmentioning

confidence: 99%

“…As the number of parameters increases, these steps become highly complex as they involve multiplication and inversion of high-dimensional matrices, whereas in the proposed SMC algorithm, the complexity increases only linearly with the number of parameters to be estimated. Previous literature, e.g., [11]- [13], also strongly mentions that not all the components of the unobserved component (UC) model are required for modeling the Web-server workload data and that the two-scale modeling is a good compromise between accuracy and computational complexity of the algorithm.…”

Section: Introductionmentioning

confidence: 99%

“…The short-term prediction is useful in predicting abnormalities in the workload data and to take care of rapid fluctuations, thereby giving the server farm management system sufficient time to prevent the possible disruption of services [11]. In this paper, in addition to the traditional short-term prediction, we also derive a scheme to predict the long-term component, utilizing the daily patterns in the workload time series.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Hierarchical Forecasting of Web Server Workload Using Sequential Monte Carlo Training

Vercauteren

Aggarwal

Wang

et al. 2006

2006 40th Annual Conference on Information Sciences and Systems

View full text Add to dashboard Cite

Abstract-Internet service utilities host multiple server applications on a shared server cluster (server farm). One of the essential tasks of the hosting service provider is to allocate servers to each of the websites to maintain a certain level of quality of service for different classes of incoming requests at each point of time, and optimize the use of server resources, while maximizing its profits. Such a proactive management of resources requires accurate prediction of workload, which is generally measured as the amount of service requests per unit time. As a time series, the workload exhibits not only short time random fluctuations but also prominent periodic (daily) patterns that evolve randomly from one period to another. We propose a solution to the Web server load prediction problem based on a hierarchical framework with multiple time scales. This framework leads to adaptive procedures that provide both longterm (in days) and short-term (in minutes) predictions with simultaneous confidence bands which accommodate not only serial correlation but also heavy-tailedness, and nonstationarity of the data. The long-term load is modeled as a dynamic harmonic regression (DHR), the coefficients of which evolve according to a random walk, and are tracked using sequential Monte Carlo (SMC) algorithms; whereas the short-term load is predicted using an autoregressive model, whose parameters are also estimated using SMC techniques. We evaluate our method using real-world Web workload data.

show abstract

Section: A Determination Of the Fixed Parametersmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Hierarchical Forecasting of Web Server Workload Using Sequential Monte Carlo Training

Vercauteren

Aggarwal

Wang

et al. 2006

2006 40th Annual Conference on Information Sciences and Systems

View full text Add to dashboard Cite

show abstract

“…Some approaches to fault detection have been developed based on Bayesian networks (e.g., [3]). In [7], the basic approach is to model the threshold metric at two levels: non-stationary behaviour as in workload forecasting and stationary behaviour with time-serial dependencies, in order to compute the probability of violations. This apparently worked well with two caveats: if the actual values of predictions were far enough from the threshold and if the prediction horizon was not too far in the future.…”

Section: Related Workmentioning

confidence: 99%

Predicting Violations of QoS Requirements in Distributed Systems

Taylor

Lutfiyya

2003

Management of Multimedia Networks and Services

View full text Add to dashboard Cite

Abstract. A Quality of Service (QoS) requirement refers to a nonfunctional requirement such as performance. A QoS management system allocates and schedules computing resources. A dynamic QoS management system is one that dynamically allocates resources to an application during its lifetime. This is usually done when the application has a QoS requirement that is not being satisfied at run-time. Ideally, the QoS management system is able to predict when the QoS requirement will be violated before it is violated. This paper describes an approach to prediction and hows shows how this was applied to a case study.

show abstract

Dynamic Resource Management

2018

Strategic Analytics

View full text Add to dashboard Cite

First of all I would like to thank my supervisor Prof. Dr. Wolfgang Nebel for his support and helpful advices concerning the principals of academic work. His contacts to industrial companies further helped me to gather some practical insights into the topic of this thesis. The theoretical concepts developed in this thesis could very closely address real practical issues due to this background. In addition, I would like to thank Prof. Dr. Michael Sonnenschein for taking the time to review this document. Much of the work presented in this thesis was supported by two of my students: Pierre Petliczew and Daniel Schlitt. Thank you both for your good work. I would also like to thank my colleagues for many constructive discussions; especially Henrik Lipskoch for helping me with mathematical background, Kiril Schröder and Daniel Schlitt for discussions about the concepts, and Domenik Helms and Gunnar Schomaker for some tips concerning the formal representation of some optimization problems. Additionally, special thanks to the NOWIS company for providing me very good evaluation data to assess my concepts. A large portion of my work was further supported by the OFFIS Institute for Information Technology. I did most of the research within the research project "Energy Efficiency in Data Centers". Finally, an internal scholarship helped me to finish my work and to write down this thesis. And last but not least I want to thank my family for their support. Especially the last two month of my work were hard for me for several reasons. Thank you for your help. *** Ein spezieller Dank soll an dieser Stelle an meine Familie für ihre mentale Unterstützung gehen. Die letzten zwei Monate dieser Arbeit waren aus verschiedenen Gründen nicht einfach für mich. Vielen Dank für Eure Hilfe. ix Performanzziele grundsätzlich nicht ausgeschlossen werden. Diese traten aber nur in weniger als 0.2% der Simulationszeit tatsächlich auf. xiii 2007 2010 2015 2020 network and telecom servers and data centers devices in business companies devices in households 55,4 58,4 63,0 year 66,7 Figure 1.1: Estimated energy consumption caused by ICT consumers in Germany[112] Consumer electronics-first and foremost the entertainment electronics-will take the biggest portion of the ICT's energy consumption in the next few years, as illustrated in Figure 1.1. But one can also note that especially data centers and the Internet will take an ever growing share due to an exponentially increasing demand. Different reasons motivate to address the high power consumption in data centers already today. Carbon dioxide emitted while electric power is generated is thought to cause ecological problems. Reducing the energy consumption may help to solve this issue. Additionally, data center operators are motivated by economical aspects. Energy cost form a significant part of the overall costs in data centers[115]. In addition, simple technical reasons need to be addressed. Some modern data centers would have the capacity to host additional servers but they do not get the power from...

show abstract

Predictive models for proactive network management: application to a production Web server

Cited by 33 publications

References 4 publications

Hierarchical Forecasting of Web Server Workload Using Sequential Monte Carlo Training

Hierarchical Forecasting of Web Server Workload Using Sequential Monte Carlo Training

Predicting Violations of QoS Requirements in Distributed Systems

Dynamic Resource Management

Contact Info

Product

Resources

About