CloudCmp

Abstract

While many public cloud providers offer pay-as-you-go computing, their varying approaches to infrastructure, virtualization, and software services lead to a problem of plenty. To help customers pick a cloud that fits their needs, we develop CloudCmp, a systematic comparator of the performance and cost of cloud providers. CloudCmp measures the elastic computing, persistent storage, and networking services offered by a cloud along metrics that directly reflect their impact on the performance of customer applications. CloudCmp strives to ensure fairness, representativeness, and compliance of these measurements while limiting measurement cost. Applying CloudCmp to four cloud providers that together account for most of the cloud customers today, we find that their offered services vary widely in performance and costs, underscoring the need for thoughtful provider selection. From case studies on three representative cloud applications, we show that CloudCmp can guide customers in selecting the best-performing provider for their applications.

On understanding the economics and elasticity challenges of deploying business applications on public cloud infrastructure

“…However, the exact initiation time is still undetermined as Amazon further states that it is dependent on a number of factors including the instance size, number of instances to be launched and how recently those instances have been launched (first time instances take longer to boot). Li et al [29] measured allocation latency and booting latency of the smallest instances, both Windows and Linux OS, for three providers including Amazon server instances. They found that all providers have average allocation latency below the 10 minutes and Windows instances take longer than Linux instances to be created and/or booted.…”

Cloud Market Maker: An automated dynamic pricing marketplace for cloud users

“…Assisting customers in deciding which cloud provider is the most suitable one for their needs is the focus of work by Ang Li et.al [13] which proposed a cloud comparator (Cloudcmp). Based on customer"s perceived preferences, a comparison of four top cloud providers including Amazon Web Services, Rackspace CloudServers, Microsoft Azure, and Google AppEngine was carried out in this paper.…”

“…Previous work [6,12,13,14] assist in the comparison of cloud providers but the decision of selection of a cloud provider is left to the customer which may be frustrating for users.…”

A price-performance analysis of EC2, Google Compute and Rackspace cloud providers for scientific computing

“…CloudCmp [20] is a framework aimed to estimate the cost and performance of legacy systems while porting and deploying it over the cloud. Three phases were involved in this process: In the service benchmarking phase, six cloud vendors (including Google AppEngine, Amazon AWS, Microsoft Azure, GoGrid and Rackspace) were selected considering the web application deployment features necessary for cloud computing services.…”