Starling: Minimizing Communication Overhead in Virtualized Computing Platforms Using Decentralized Affinity-Aware Migration

Abstract

Virtualization is being widely used in large-scale computing environments, such as clouds, data centers, and grids, to provide application portability and facilitate resource multiplexing while retaining application isolation. In many existing virtualized platforms, it has been found that the network bandwidth often becomes the bottleneck resource, causing both high network contention and reduced performance for communication and data-intensive applications. In this paper, we present a decentralized affinity-aware migration technique that incorporates heterogeneity and dynamism in network topology and job communication patterns to allocate virtual machines on the available physical resources. Our technique monitors network affinity between pairs of VMs and uses a distributed bartering algorithm, coupled with migration, to dynamically adjust VM placement such that communication overhead is minimized. Our experimental results running the Intel MPI benchmark and a scientific application on a 7-node Xen cluster show that we can get up to 42% improvement in the runtime of the application over a no-migration technique, while achieving up to 85% reduction in network communication cost. In addition, our technique is able to adjust to dynamic variations in communication patterns and provides both good performance and low network contention with minimal overhead. We also present a topology-aware extension to our migration algorithm that provides an additional 26-31% reduction in runtime.

Remedy: Network-Aware Steady State VM Management for Data Centers

“…Second, we consider the cost of VM migration, and ensure that the migration is completed in a timely manner, which are not considered in [7]. Sonnek et al [14] propose a technique that monitors network affinity between pairs of VMs and uses a distributed bartering algorithm coupled with migration to dynamically adjust VM placement such that communication overhead is minimized. Meng et al [13] consider heuristics for VM placement in a data center to optimize network traffic while minimizing the cost of VM migration.…”

Achieving Intelligent Traffic-Aware Consolidation of Virtual Machines in a Data Center Using Learning Automata

“…From the results provided, one can conclude that the function value given by the Cluster-and-Cut algorithm is ⇠10% smaller than the measures obtained by the two benchmarks. Starling: Minimizing Communication Using Decentralized Affinity-Aware Migration: Sonnek et al [36] introduce a decentralized affinity-aware migration technique for allocating VMs on the available physical resources. The technique monitors the network affinity between the pairs of VMs and uses a distributed bartering algorithm together with VM migration in order to dynamically move VMs in a way that ensures that the communication overhead is minimized.…”

“…The goal of this paper is to see how the VMs can be optimally placed within a data center in a traffic-aware manner. Viewed from a traffic-aware perspective, the resource of bandwidth becomes a bottleneck in the higher layers of the network, decreasing the performance when it concerns communication [36] between the applications. This also increases the workload for the network elements on the aggregation and core layers, which, in turn, often results in higher power consumption within the data center [13], more greenhouse emissions, and the increased business costs.…”

Replica placement to mitigate attacks on clouds

“…Popular strategies such as replication may not work well on the cloud, because providers may take advantage of virtualization techniques [1][2][3][4][5] to concentrate some of (or all) the replicas in the same physical machine (PM). Recent research has explored affinities among virtual machines (VMs) to consolidate according to traffic [6,7] or memory pages [8,9], for example. We could think of a cluster of application servers responding to HTTP requests, or a remote storage service [10][11][12].…”