Universality of Power-of-d Load Balancing in Many-Server Systems

Abstract

We consider a system of N parallel single-server queues with unit exponential service rates and a single dispatcher where tasks arrive as a Poisson process of rate λ(N). When a task arrives, the dispatcher assigns it to a server with the shortest queue among d(N) randomly selected servers (1 d(N) N). This load balancing strategy is referred to as a JSQ(d(N)) scheme, marking that it subsumes the celebrated Join-the-Shortest Queue (JSQ) policy as a crucial special case for d(N) = N.We construct a stochastic coupling to bound the difference in the queue length processes between the JSQ policy and a JSQ(d(N)) scheme with an arbitrary value of d(N). We use the coupling to derive the fluid limit in the regime where λ(N)/N → λ < 1 as N → ∞ with d(N) → ∞, along with the associated fixed point. The fluid limit turns out not to depend on the exact growth rate of d(N), and in particular coincides with that for the JSQ policy. We further leverage the coupling to establish that the diffusion limit in the critical regime where (N − λ(N))/ √ N → β > 0 as N → ∞ with d(N)/( √ N log(N)) → ∞ corresponds to that for the JSQ policy. These results indicate that the optimality of the JSQ policy can be preserved at the fluid-level and diffusion-level while reducing the overhead by nearly a factor O(N) and O( √ N/ log(N)), respectively. * d.mukherjee@tue.nl

Job assignment in large-scale service systems with affinity relations

“…for i ≥ 0 [14]. This shows a strong similarity with the fixed point (16) where two types of jobs are taken into account. Next we consider the case d 1 = 1.…”

“…Fixed points with q 00 = 0. The correctness of the expression in (16) can easily be confirmed by substitution into (15). The result can be established in two steps.…”

“…Letλ be as defined in (14) for the fixed point (16). Then the long-term fraction of servers with at least i jobs under a JSQ(d 1 ) policy where each server works at rate µ 1 is given by…”

“…d 1 ≥ d * 1 . From Theorem 4.1 we know that, next to the closed form fixed point (16), there are two additional fixed points with q 00 + q 01 + q 01 = 1. We prove the following theorem using the indirect Lyapunov method.…”

“…In total this gives rise to two locally stable fixed points: the closed-form fixed point (16) where each server has a type II job and possibly multiple type I jobs and the fixed point from Theorem 4.2 where at most one job is present at each server. In the remainder of this section we will refer to these fixed points as the queueing fixed point and no-queueing fixed point, respectively.…”