On the Performance Variability of Production Cloud Services

Abstract

Abstract-Cloud computing is an emerging infrastructure paradigm that promises to eliminate the need for companies to maintain expensive computing hardware. Through the use of virtualization and resource time-sharing, clouds address with a single set of physical resources a large user base with diverse needs. Thus, clouds have the potential to provide their owners the benefits of an economy of scale and, at the same time, become an alternative for both the industry and the scientific community to self-owned clusters, grids, and parallel production environments. For this potential to become reality, the first generation of commercial clouds need to be proven to be dependable. In this work we analyze the dependability of cloud services. Towards this end, we analyze long-term performance traces from Amazon Web Services and Google App Engine, currently two of the largest commercial clouds in production. We find that the performance of about half of the cloud services we investigate exhibits yearly and daily patterns, but also that most services have periods of especially stable performance. Last, through tracebased simulation we assess the impact of the variability observed for the studied cloud services on three large-scale applications, job execution in scientific computing, virtual goods trading in social networks, and state management in social gaming. We show that the impact of performance variability depends on the application, and give evidence that performance variability can be an important factor in cloud provider selection.

Deployment Strategies for Distributed Applications on Cloud Computing Infrastructures

“…A big variance in node creation times leads to a higher potential speed gain. Our experimental results show that there is indeed a substantial difference between node creation times, and this is supported by other experiments [16]. Also, a large difference between average software deployment times is beneficial, e.g., the installation of software A needs to take much more or much less time, on average, than that of software B.…”

“…Others have also studied the performance of public cloud services, focusing on the startup time of VMs [15], tracking performance of several operations within a cloud environment [16], or on the prediction of application performance [17]. This paper adds the metrics of individual deployment steps, such as the installation, configuration and starting of software.…”

DynamicCloudSim: Simulating heterogeneity in computational clouds

“…The effect of these factors of uncertainty and instability is not negligible and has been repeatedly observed to strongly influence the runtime of a given application on commercial clouds such as Amazon's Elastic Compute Cloud (EC2) (e.g., [7][8][9][10][11][12][13]). …”

Scalable Architectures for Platform-as-a-Service Clouds: Performance and Cost Analysis

“…Ours takes costs into account and evaluates the trade-off, while theirs links scalability to security concerns. In a similar vein, [7] focusses on performance. These works consider earlier pre-cloud investigations as in [9] or [10].…”