Automatic virtual machine clustering based on bhattacharyya distance for multi-cloud systems

Abstract

Size and complexity of modern data centers pose scalability issues for the resource monitoring system supporting management operations, such as server consolidation. When we pass from cloud to multi-cloud systems, scalability issues are exacerbated by the need to manage geographically distributed data centers and exchange monitored data across them. While existing solutions typically consider every Virtual Machine (VM) as a black box with independent characteristics, we claim that scalability issues in multi-cloud systems could be addressed by clustering together VMs that show similar behaviors in terms of resource usage. In this paper, we propose an automated methodology to cluster VMs starting from the usage of multiple resources, assuming no knowledge of the services executed on them. This innovative methodology exploits the Bhattacharyya distance to measure the similarity of the probability distributions of VM resources usage, and automatically selects the most relevant resources to consider for the clustering process. The methodology is evaluated through a set of experiments with data from a cloud provider. We show that our proposal achieves high and stable performance in terms of automatic VM clustering. Moreover, we estimate the reduction in the amount of data collected to support system management in the considered scenario, thus showing how the proposed methodology may reduce the monitoring requirements in multi-cloud systems.

Balancing Accuracy and Execution Time for Similar Virtual Machines Identification in IaaS Cloud

“…The use of this distance for VM clustering has been proposed for the first time in a preliminary paper [15].…”

“…In this experiment we evaluate the sensitivity to the number of metrics on the clustering techniques by considering a reduced set of metrics, which is limited to four metrics mostly used in data center management strategies [4], [6], [18]: CPU and memory utilization, input and output packet rate. It is worth to note that an automatic mechanism to select metrics for VM clustering purposes has been proposed in a preliminary study by the authors [15]: the selection, which is based on the analysis of autocorrelation and coefficients of variation of the time series, confirms the presence of the above mentioned metrics in the selected set. Figure 3 shows the purity of the clustering approaches for the entire set of 10 metrics and for the reduced set of four selected metrics.…”

An adaptive technique to model virtual machine behavior for scalable cloud monitoring

“…We now describe the reference scenario for our proposal, where a IaaS cloud data center integrates a clustering technique [1], [2] to improve the scalability of monitoring and management. Throughout this section, we outline the main issues of this approach, namely cluster-based monitoring and management, and motivate the proposal of an adaptive methodology to model VMs behavior that aims to solve these problems.…”

“…Recent research studies [1], [2] show that automatically clustering VMs with similar behaviors in terms of resource usage may improve the scalability of the monitoring process in IaaS cloud systems. The identification of classes of VMs behaving in the same way allows to reduce the amount of globally collected data, limiting a fine-grained monitoring to few representatives for each class, and performing a coarsegrained monitoring on the other VMs.…”

“…The identification of classes of VMs behaving in the same way allows to reduce the amount of globally collected data, limiting a fine-grained monitoring to few representatives for each class, and performing a coarsegrained monitoring on the other VMs. However, the solutions presented in [1], [2] show a clear trade-off between the VM clustering accuracy (that is the fraction VMs assigned to the correct class) and the length of the resource usage time series used for clustering. Specifically, long time series (up to weeks of collected measurements) should achieve a clustering that correctly groups every VMs.…”