Hierarchical Forecasting of Web Server Workload Using Sequential Monte Carlo Training

Abstract

Abstract-Internet service utilities host multiple server applications on a shared server cluster (server farm). One of the essential tasks of the hosting service provider is to allocate servers to each of the websites to maintain a certain level of quality of service for different classes of incoming requests at each point of time, and optimize the use of server resources, while maximizing its profits. Such a proactive management of resources requires accurate prediction of workload, which is generally measured as the amount of service requests per unit time. As a time series, the workload exhibits not only short time random fluctuations but also prominent periodic (daily) patterns that evolve randomly from one period to another. We propose a solution to the Web server load prediction problem based on a hierarchical framework with multiple time scales. This framework leads to adaptive procedures that provide both longterm (in days) and short-term (in minutes) predictions with simultaneous confidence bands which accommodate not only serial correlation but also heavy-tailedness, and nonstationarity of the data. The long-term load is modeled as a dynamic harmonic regression (DHR), the coefficients of which evolve according to a random walk, and are tracked using sequential Monte Carlo (SMC) algorithms; whereas the short-term load is predicted using an autoregressive model, whose parameters are also estimated using SMC techniques. We evaluate our method using real-world Web workload data.

A predictive and probabilistic load-balancing algorithm for cluster-based web servers

“…Many dynamic load-balancing algorithms collect instantaneous load status of web servers such as CPU load, disk usage and number of active network connections as load descriptor [1,2,3,5,28,29,33,35,40,48,51,52,54,56]. The load status which is obtained via direct measurement has fluctuations at different time scales, and becomes obsolete quickly [28,33,35].…”

“…Therefore, a decision which is made based on direct resource measurements (e.g., the CPU load or the average web object response time), may be risky if not completely wrong. Some previous works [6,10,52] have used linear models to estimate real loads via measured resources. Linear models work well when measured values are highly correlated, which is not always the case in web server systems with heavy-tail model of traffics.…”

“…However, as the prevalence of web in business and critical domains grows, it is becoming evident that the web servers must be made with superior performance, in order to retain existing users and attract new ones [4,6,52,54]. One of the most popular approaches for addressing these challenges is the cluster-based web servers [16,20,28,29,44,51,61].…”

An approximation-based load-balancing algorithm with admission control for cluster web servers with dynamic workloads

“…This shortcoming was improved in the second generation of load-balancing algorithms such as Weighted Round-Robin (WRR), Least Connections (LC) and Weighted Least Connections (WLC) [12,40]. The improved algorithms collect the instantaneous load status of web servers such as CPU load, disk usage and number of active network connections as a load descriptor [2,4,5,7,10,12,13,16,26,40,46,48] and use this information for the server selection decisions. The communication cost of the load measurements in this method is relatively high, specifically when a large number of nodes are used in the cluster.…”

“…Therefore, direct resource measurements do not show the real load status of web servers. Some previous works [22,48] have used linear models to estimate the actual server loads via direct resource measurements. But linear models work well when measured values are highly correlated, which is not always the case in web server systems with a heavy-tail model of the traffic.…”