Many-server asymptotics for Join-the-Shortest Queue in the Super-Halfin-Whitt Scaling Window

Abstract

The Join-the-Shortest Queue (JSQ) policy is a classical benchmark for the performance of many-server queueing systems due to its strong optimality properties. While the exact analysis of the JSQ policy is an open question to date, even under Markovian assumption on the service requirements, recently, there has been a significant progress in understanding its manyserver asymptotic behavior since the work of Eschenfeldt and Gamarnik (Math. Oper. Res. 43 (2018) 867-886).In this paper, we analyze the many-server limits of the JSQ policy in the super-Halfin-Whitt scaling window when load per server λ N scales with the system size N as lim N→∞ N α (1 − λ N ) = β for α ∈ (1/2, 1) and β > 0. We establish that the centered and scaled total queue length process converges to a certain Bessel process with negative drift and the associated centered and scaled steady-state total queue length, indexed by N, converges to a Gamma(2, β) distribution. Both the transient and steady-state limit laws are universal in the sense that they do not depend on the value of the scaling parameter α, and exhibit fundamentally different qualitative behavior from both the Halfin-Whitt regime (α = 1/2) and the Non-degenerate Slowdown (NDS) regime (α = 1).

Large-System Insensitivity of Zero-Waiting Load Balancing Algorithms

“…Significant processes have been made over the past few years on understanding achieving asymptotic zero-waiting (as the system size approaches infinity) in a large-scale data center with distributed queues, including the classic supermarket model [14,8,32,17,3,4,30,24,25,23,22,45,9], models with data locality [40,31] and models where each job consists of parallel tasks [39,37,19], etc.…”

“…However, almost all these results assume exponential service time distributions. While each of these results [14,8,32,17,29,30,24,25,23,22,40,31,39,37,19,45,9] provided important insights of achieving zero-waiting in a practical system, theoretically, it is not clear whether these principles hold for general service times. This is a very important question to answer because it is well-known that service time distributions in real-world systems are not exponential.…”