Asymptotic Optimality of Power-of-$d$ Load Balancing in Large-Scale Systems

Mukherjee, Debankur; Borst, Sem C.; van Leeuwaarden, Johan S. H.; Whiting, Philip A.

doi:10.48550/arxiv.1612.00722

Cited by 7 publications

(13 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The above results mirror the fluid-level and diffusion-level optimality properties reported in the companion paper [23] for power-of-d(N) strategies in a scenario with N server pools, where each server pool is a collection of servers, each working at unit rate. The coupling developed in [23] has greater hold on the task completions, and provides absolute bounds on the difference of each component of the occupancy states. More specifically, the task completions depend on the total number of active tasks in the entire system, whereas in the single-server scenario, it depends only on the number of non-idle servers.…”

Section: Introductionsupporting

confidence: 77%

“…The above condition is in fact close to necessary, in the sense that the diffusion-level behavior of the scheme is sub-optimal if d(N)/( √ N log N) → 0 as N → ∞. The above results mirror the fluid-level and diffusion-level optimality properties reported in the companion paper [23] for power-of-d(N) strategies in a scenario with N server pools, where each server pool is a collection of servers, each working at unit rate. The coupling developed in [23] has greater hold on the task completions, and provides absolute bounds on the difference of each component of the occupancy states.…”

Section: Introductionsupporting

confidence: 73%

See 1 more Smart Citation

Universality of Power-of-d Load Balancing in Many-Server Systems

et al. 2018

Self Cite

View full text Add to dashboard Cite

We consider a system of N parallel single-server queues with unit exponential service rates and a single dispatcher where tasks arrive as a Poisson process of rate λ(N). When a task arrives, the dispatcher assigns it to a server with the shortest queue among d(N) randomly selected servers (1 d(N) N). This load balancing strategy is referred to as a JSQ(d(N)) scheme, marking that it subsumes the celebrated Join-the-Shortest Queue (JSQ) policy as a crucial special case for d(N) = N.We construct a stochastic coupling to bound the difference in the queue length processes between the JSQ policy and a JSQ(d(N)) scheme with an arbitrary value of d(N). We use the coupling to derive the fluid limit in the regime where λ(N)/N → λ < 1 as N → ∞ with d(N) → ∞, along with the associated fixed point. The fluid limit turns out not to depend on the exact growth rate of d(N), and in particular coincides with that for the JSQ policy. We further leverage the coupling to establish that the diffusion limit in the critical regime where (N − λ(N))/ √ N → β > 0 as N → ∞ with d(N)/( √ N log(N)) → ∞ corresponds to that for the JSQ policy. These results indicate that the optimality of the JSQ policy can be preserved at the fluid-level and diffusion-level while reducing the overhead by nearly a factor O(N) and O( √ N/ log(N)), respectively. * d.mukherjee@tue.nl

show abstract

Section: Introductionsupporting

confidence: 77%

Section: Introductionsupporting

confidence: 73%

Universality of Power-of-d Load Balancing in Many-Server Systems

et al. 2018

Self Cite

View full text Add to dashboard Cite

show abstract

“…Now, choosing n(N) = √ N/ d(N)/( √ N log(N)), it can be seen that as N → ∞, n(N)/ √ N → 0 and the above probability converges to 0. Therefore for any ε, δ > 0, (19) yields…”

Section: Asymptotically Optimal Random Graph Topologiesmentioning

confidence: 98%

“…Our proof methodology builds on some recent advances in the analysis of the power-of-d algorithm where d = d(N) grows with N [19,20]. Specifically, we view the load balancing process on an arbitrary graph as a 'sloppy' version of that on a clique, and thus construct several other intermediate sloppy versions.…”

Section: Related Workmentioning

confidence: 99%

Asymptotically Optimal Load Balancing Topologies

Mukherjee

Borst

Leeuwaarden

2018

SIGMETRICS Perform. Eval. Rev.

Self Cite

View full text Add to dashboard Cite

We consider a system of N servers inter-connected by some underlying graph topology G N . Tasks with unit-mean exponential processing times arrive at the various servers as independent Poisson processes of rate λ. Each incoming task is irrevocably assigned to whichever server has the smallest number of tasks among the one where it appears and its neighbors in G N .The above model arises in the context of load balancing in large-scale cloud networks and data centers, and has been extensively investigated in the case G N is a clique. Since the servers are exchangeable in that case, mean-field limits apply, and in particular it has been proved that for any λ < 1, the fraction of servers with two or more tasks vanishes in the limit as N → ∞. For an arbitrary graph G N , mean-field techniques break down, complicating the analysis, and the queue length process tends to be worse than for a clique. Accordingly, a graph G N is said to be N-optimal or √ N-optimal when the queue length process on G N is equivalent to that on a clique on an N-scale or √ N-scale, respectively. We prove that if G N is an Erdős-Rényi random graph with average degree d(N), then with high probability it is N-optimal and √ N-optimal if d(N) → ∞ and d(N)/( √ N log(N)) → ∞ as N → ∞, respectively. This demonstrates that optimality can be maintained at N-scale and √ N-scale while reducing the number of connections by nearly a factor N and √ N/ log(N) compared to a clique, provided the topology is suitably random. It is further shown that if G N contains Θ(N) bounded-degree nodes, then it cannot be N-optimal. In addition, we establish that an arbitrary graph G N is N-optimal when its minimum degree is N − o(N), and may not be N-optimal even when its minimum degree is cN + o(N) for any 0 < c < 1/2. Simulation experiments are conducted for various scenarios to corroborate the asymptotic results. * d.mukherjee@tue.nl; † s.c.borst@tue.nl; ‡ j.s.h.v.leeuwaarden@tue.nl arXiv:1707.05866v2 [math.PR] 6 Apr 2019Related work. The above model has been studied in [11,28], focusing on certain fixed-degree graphs and in particular ring topologies. The results demonstrate that the flexibility to forward tasks to a few neighbors, or even just one, with possibly shorter queues significantly improves the performance in terms of the waiting time and tail distribution of the queue length. This resembles the so-called 'power-of-two' effect in the classical case of a complete graph where tasks are assigned to the shortest queue among d servers selected uniformly at random. As shown by Mitzenmacher [16,17] and Vvedenskaya et al. [31], such a 'power-of-d' scheme provides a huge performance improvement over purely random assignment, even when d = 2, in particular super-exponential tail decay, translating into far better waiting-time performance. Further related

show abstract

“…Similarly as in the blocking scenario, the term z r (t) represents the (scaled) rate at which dispatcher r uses tokens and forwards incoming jobs to idle servers at time t. Accordingly, λ 1 (t) is the aggregate rate at which dispatchers use tokens to forward jobs to (guaranteed) idle servers at time t, while λ 2 (t) is the aggregate rate at which jobs are forwarded to randomly selected servers (which may or may not be idle). Equation (10) reflects that the rate of change in the fraction of idle servers is the difference between the aggregate rate y 0 (t) at which jobs are completed by servers with one job, and the rate λ 1 (t) at which dispatchers use tokens to forward jobs to idle servers plus the rate λ 2 (t)y 0 (t) at which jobs are forward to randomly selected servers that happen to be idle. Equation (11) states that the rate of change in the fraction of servers with i jobs is the balance of the rate λ 2 (t)y i−1 (t) at which jobs are forwarded to randomly selected servers with i−1 jobs plus the aggregate rate y i+1 (t) at which jobs are completed by servers with i + 1 jobs, and the rate λ 2 (t)y i (t) at which jobs are forwarded to randomly selected servers with i jobs plus the aggregate rate y i (t) at which jobs are completed by servers with i jobs.…”

Section: Fluid Limit In the Queueing Scenariomentioning

confidence: 99%

Load balancing in large-scale systems with multiple dispatchers

Boor

Borst

Leeuwaarden

2017

IEEE INFOCOM 2017 - IEEE Conference on Computer Communications

View full text Add to dashboard Cite

Abstract-Load balancing algorithms play a crucial role in delivering robust application performance in data centers and cloud networks. Recently, strong interest has emerged in Jointhe-Idle-Queue (JIQ) algorithms, which rely on tokens issued by idle servers in dispatching tasks and outperform power-of-d policies. Specifically, JIQ strategies involve minimal information exchange, and yet achieve zero blocking and wait in the manyserver limit. The latter property prevails in a multiple-dispatcher scenario when the loads are strictly equal among dispatchers. For various reasons it is not uncommon however for skewed load patterns to occur. We leverage product-form representations and fluid limits to establish that the blocking and wait then no longer vanish, even for arbitrarily low overall load. Remarkably, it is the least-loaded dispatcher that throttles tokens and leaves idle servers stranded, thus acting as bottleneck.Motivated by the above issues, we introduce two enhancements of the ordinary JIQ scheme where tokens are either distributed non-uniformly or occasionally exchanged among the various dispatchers. We prove that these extensions can achieve zero blocking and wait in the many-server limit, for any subcritical overall load and arbitrarily skewed load profiles. Extensive simulation experiments demonstrate that the asymptotic results are highly accurate, even for moderately sized systems.

show abstract

Asymptotic Optimality of Power-of-$d$ Load Balancing in Large-Scale Systems

Cited by 7 publications

References 0 publications

Universality of Power-of-d Load Balancing in Many-Server Systems

Universality of Power-of-d Load Balancing in Many-Server Systems

Asymptotically Optimal Load Balancing Topologies

Load balancing in large-scale systems with multiple dispatchers

Contact Info

Product

Resources

About