Large amount of monitoring data can be collected from distributed systems as the observables to analyze system behaviors. However, without reasonable models to characterize systems, we can hardly interpret such monitoring data effectively for system management. In this paper, a new concept named flow intensity is introduced to measure the intensity with which internal monitoring data reacts to the volume of user requests in distributed transaction systems. We propose a novel approach to automatically model and search relationships between the flow intensities measured at various points across the system. If the modeled relationships hold all the time, they are regarded as invariants of the underlying system. Experimental results from a real system demonstrate that such invariants widely exist in distributed transaction systems. Further we discuss how such invariants can be used to characterize complex systems and support autonomic system management.
Abstract-In this paper, we undertake the problem of server consolidation in virtualized data centers from the perspective of approximation algorithms. We formulate server consolidation as a stochastic bin packing problem, where the server capacity and an allowed server overflow probability p are given, and the objective is to assign VMs to as few physical servers as possible, and the probability that the aggregated load of a physical server exceeds the server capacity is at most p.We propose a new VM sizing approach called effective sizing, which simplifies the stochastic optimization problem by associating a VM's dynamic load with a fixed demand. Effective sizing decides a VM's resource demand through statistical multiplexing principles, which consider various factors impacting the aggregated resource demand of a host where the VM may be placed. Based on effective sizing, we design a suite of polynomial time VM placement algorithms for both VM migration cost-oblivious and migration cost-aware scenarios.Through analysis, we show that our algorithm is O(1)-approximation for the stochastic bin packing problem when the VM loads can be modeled as all Poisson or all normal distributions. Through evaluations driven by a real data center load trace, we show that our consolidation solution can achieve an order of reduction on physical server requirement compared to that before consolidation; the consolidation result is only 24% more than the optimal solution. With effective sizing, our server consolidation solution achieves 10% to 23% more energy savings than state-of-the-art approaches.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.