SpotLight: An Information Service for the Cloud

Abstract

Infrastructure-as-a-Service cloud platforms are incredibly complex: they rent hundreds of different types of servers across multiple geographical regions under a wide range of contract types that offer varying tradeoffs between risk and cost. Unfortunately, the internal dynamics of cloud platforms are opaque in several dimensions. For example, while the risk of servers not being available when requested is critical in optimizing these risk-cost tradeoffs, it is not typically made visible to users. Thus, inspired by prior work on Internet bandwidth probing, we propose actively probing cloud platforms to explicitly learn such information, where each "probe" is a request for a particular type of server. We model the relationships between different contracts types to develop a market-based probing policy, which leverages the insight that real-time prices in cloud spot markets loosely correlate with the supply (and availability) of fixed-price on-demand servers. That is, the higher the spot price for v a server, the more likely the corresponding fixed-price on-demand server is not available. We incorporate market-based probing into SpotLight, an information service that enables cloud applications to query this and other data, and use it to monitor the availability of more than 4500 distinct server types across 9 geographical regions in Amazon's Elastic Compute Cloud over a 3 month period. We analyze this data to reveal interesting observations about the platform's internal dynamics. We then show how SpotLight enables two recently proposed derivative cloud services to select a better mix of servers to host applications, which improves their availability from

Speeding up Deep Learning with Transient Servers

“…Here the availability of cloud servers refers to the probability of cloud providers fulfilling the resource request in a timely manner. Availability depends, in part, on the overall demand for servers (both on-demand and transient) in the local region [11]. Therefore, to best utilize transient servers it is likely that customers will need to request servers with different (but more available) resource capacities and from multiple regions.…”

Statistical analysis of Amazon EC2 cloud pricing models

“…SpotLight aims to help cloud provider managers and cloud users to better understand the dynamics of cloud pricing policies. The authors propose an active probing approach to monitor the availability of on demand and spot Amazon EC2 instances.…”

Portfolio-driven Resource Management for Transient Cloud Servers

“…The lack of variable pricing means that we cannot use it to estimate the correlations between different server types. Given the lack of any availability data, we assume that ExoSphere would require active probing [39] to infer availability. For our experiments, we use the covariance matrix from the EC2 markets.…”

“…In addition, the availability of transient servers (in terms of their mean time to revocation), can also vary significantly across server configurations and based on changing market conditions. Unfortunately, cloud platforms generally do not directly expose availability statistics for transient servers, requiring users to indirectly infer them, e.g., via a price history or active probing [39]. Thus, it is challenging workload and risk tolerance.…”