Cloud control with distributed rate limiting

Abstract

Today's cloud-based services integrate globally distributed resources into seamless computing platforms. Provisioning and accounting for the resource usage of these Internet-scale applications presents a challenging technical problem. This paper presents the design and implementation of distributed rate limiters, which work together to enforce a global rate limit across traffic aggregates at multiple sites, enabling the coordinated policing of a cloud-based service's network traffic. Our abstraction not only enforces a global limit, but also ensures that congestion-responsive transport-layer flows behave as if they traversed a single, shared limiter. We present two designs-one general purpose, and one optimized for TCP-that allow service operators to explicitly trade off between communication costs and system accuracy, efficiency, and scalability. Both designs are capable of rate limiting thousands of flows with negligible overhead (less than 3% in the tested configuration). We demonstrate that our TCP-centric design is scalable to hundreds of nodes while robust to both loss and communication delay, making it practical for deployment in nationwide service providers.

Dynamically scaling applications in the cloud

“…Thisà la cloud network provision paradigm can be supported by flow control [29], distributed rate limiting [30], and network slicing techniques [31]. By applying this mechanism the actual bandwidth can be dynamically allocated to applications on demand, which would benefit from a dynamic resource allocation scheme in which all the users pay for the actual bandwidth consumption.…”

“…Network slicing [29,17,31,30] Keep separate per application flows by adapting to on demand network utilization needs by every application, dynamic network bandwidth allocation.…”

Distributed resource management across process boundaries

“…[23,33,44,75] are potential admission control policies for Wisp. [59] focuses on the design of distributed rate limiters for network flows. Themis [40] manages overload for federated stream processing systems by degrading query quality fairly across users, presenting a potential rate limiting policy for Wisp.…”

Bandwidth adaptation in streaming overlays

“…There exist commercial solutions which can limit bandwidth on a per-site basis [21], but these cannot enforce a limit over a set of wide-area network locations. Recently Raghavan et al [24] described methods for controlling the bandwidth consumption of an ensemble of flows which do not have to pass through the same infrastructure. Although these approaches can limit the amount of bandwidth an organization devotes to an overlay service, they are not appropriate for a streaming service as a large fraction of nodes might experience interrupted playback when rate limiting takes place.…”