Image Distribution Mechanisms in Large Scale Cloud Providers

Wartel, Romain; Cass, Tony; Moreira, Belmiro; Roche, Ewan; Guijarro, Manuel; Goasguen, Sebastien; Schwickerath, U.

doi:10.1109/cloudcom.2010.73

Cited by 46 publications

(36 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When this is multiplied by the number of hosts which can be tens or even hundreds, the amount of data to be transferred becomes significant. Many different methods for achieving this task have been discussed in previous literature [3][4][5][6][7][8][9]. We classify these methods into four categories: centralized concurrent unicast, multicast, peer-assisted and on-demand distribution.…”

Section: Virtual Machine Deployment Problemmentioning

confidence: 99%

See 1 more Smart Citation

Improving the Efficiency of Deploying Virtual Machines in a Cloud Environment

Laurikainen

Laitinen

Lehtovuori

et al. 2012

2012 International Conference on Cloud and Service Computing

View full text Add to dashboard Cite

Flexible allocation of resources is one of the main benefits of cloud computing. Virtualization is used to achieve this flexibility: one or more virtual machines run on a single physical machine. These virtual machines can be deployed and destroyed as needed. One obstacle to flexibility in current cloud systems is that deploying multiple virtual machines simultaneously on multiple physical machines is slow due to the inefficient usage of available resources.We implemented and evaluated three methods of transferring virtual machine images for the OpenNebula cloud middleware. One of the implementations was based on BitTorrent and the other two were based on multicast. Our evaluation results showed that the implemented methods were significantly more scalable than the default methods available in OpenNebula when tens of virtual machines were deployed simultaneously. However, the implemented methods were slightly slower than the old ones for deploying only one or a few virtual machines at a time due to overhead related to managing the transfer process.We also evaluated the performance of different virtual machine disk formats, as this choice also affects the deployment time of the machine. Raw images, QCOW2 images and logical volumes were evaluated. Logical volumes were fastest overall in sequential disk I/O performance. With sequential reads and writes, raw images could provide at best approximately 88% of the write performance and 95% of the read performance of logical volumes. The corresponding numbers for QCOW2 were 86% write and 74% read performance. Random access performance between QCOW2 and raw images was nearly identical, but LVM random access performance in our specific benchmark was significantly worse.If the usage pattern of the cloud is such that deploying large batches of virtual machines at once is common, using the new transfer methods will significantly speed up the deployment process and reduce its resource usage. The disk access method should be chosen based on what provides acceptable performance for the task being executed and provides the fastest deployment times.

show abstract

Section: Virtual Machine Deployment Problemmentioning

confidence: 99%

“…Peer-assisted distribution Another approach is to divide the task of serving the image among the physical hosts involved in the deployment [5,6]. There are two basics ways of doing this.…”

Section: Virtual Machine Deployment Problemmentioning

confidence: 99%

Improving the Efficiency of Deploying Virtual Machines in a Cloud Environment

Laurikainen

Laitinen

Lehtovuori

et al. 2012

2012 International Conference on Cloud and Service Computing

View full text Add to dashboard Cite

show abstract

“…However, the broadcast step has an high overhead both in execution time and network traffic, which reduces the attractiveness of IaaS for short-lived jobs and is expensive for the provider (in terms of lost resources that could otherwise be charged for). Thus, reducing the broadcast overhead has been an active area of study, with proposals ranging from multi-cast [9] and application level broadcast-trees [10] to peer-to-peer protocols [11,4].…”

Section: Related Workmentioning

confidence: 99%

“…Current techniques often pre-copy the full VM image locally on the compute nodes before launching the VM instances, which can take in the order of tens of minutes or even hours [4], not counting the time to boot the guest operating system and deploy the application itself. Although on-demand techniques have matured (e.g., locally derived copy-on-write images [5] that use a remotely stored VM image template as a backing file) and they have been shown to generate little overhead on application performance compared to the case when a local copy is available [6], they saw comparatively little attention for multi-deployments due to the fact that they generate I/O contention to the repository where the VM image template is stored.…”

Section: Introductionmentioning

confidence: 99%

Leveraging Collaborative Content Exchange for On-Demand VM Multi-deployments in IaaS Clouds

Nicolae

Rafique

2013

Euro-Par 2013 Parallel Processing

View full text Add to dashboard Cite

Abstract. A critical feature of IaaS cloud computing is the ability to deploy, boot and terminate large groups of inter-dependent VMs very quickly, which enables users to efficiently exploit the on-demand nature and elasticity of clouds even for large-scale deployments. A common pattern in this context is multi-deployment, i.e., using the same VM image template to instantiate a large number of VMs in parallel. A difficult trade-off arises in this context: access the content of the template on-demand but slowly due to I/O bottlenecks or pre-broadcast the full contents of the template on the local storage of the hosting nodes to avoid such bottlenecks. Unlike previous approaches that are biased towards either of the extremes, we propose a scheme that augments on-demand access through a collaborative scheme in which the VMs aim to leverage the similarity of access pattern in order to anticipate future accesses and exchange chunks between themselves in an attempt to reduce contention to the remote storage where the VM image template is stored. Large scale experiments show improvements in read throughput between 30%-40% compared to on-demand access schemes that perform in isolation.

show abstract

“…In the first scenario, a single VMI needs to be transferred to many physical nodes. Here, peer-to-peer networking is a commonly used technique [3,17,20]. SnowFlock [11] clones VMs from already running ones in less than one second.…”

Section: Efficient Vmi Transfermentioning

confidence: 99%

Reducing VM Startup Time and Storage Costs by VM Image Content Consolidation

Razavi

Razorea

Kielmann

2014

Euro-Par 2013: Parallel Processing Workshops

View full text Add to dashboard Cite

Abstract. Elastic cloud applications rely on fast virtual machine (VM) startup, e.g. when scaling out for handling increased workload. While there have been recent studies into the VM startup time in clouds, the effects of the VM image (VMI) disk size and its contents are little understood. To fill this gap, we present a detailed study of these factors on Amazon EC2. Based on our findings, we developed a novel approach for consolidating size and contents of VMIs. We then evaluated our approach with the ConPaaS VMI, an open-source Platform-as-a-Service runtime. Compared to an unmodified ConPaaS VMI, our approach results in up to four times reduction of the disk size, three times speedup for the VM startup time, and three times reduction of storage cost.

show abstract

Image Distribution Mechanisms in Large Scale Cloud Providers

Cited by 46 publications

References 11 publications

Improving the Efficiency of Deploying Virtual Machines in a Cloud Environment

Improving the Efficiency of Deploying Virtual Machines in a Cloud Environment

Leveraging Collaborative Content Exchange for On-Demand VM Multi-deployments in IaaS Clouds

Reducing VM Startup Time and Storage Costs by VM Image Content Consolidation

Contact Info

Product

Resources

About