Zero Queueing for Multi-Server Jobs

Abstract

Cloud computing today is dominated by multi-server jobs. These are jobs that request multiple servers simultaneously and hold onto all of these servers for the duration of the job. Multi-server jobs add a lot of complexity to the traditional one-server-per-job model: an arrival might not "fit" into the available servers and might have to queue, blocking later arrivals and leaving servers idle. From a queueing perspective, almost nothing is understood about multiserver job queueing systems; even understanding the exact stability region is a very hard problem.In this paper, we investigate a multi-server job queueing model under scaling regimes where the number of servers in the system grows. Specifically, we consider a system with multiple classes of jobs, where jobs from different classes can request different numbers of servers and have different service time distributions, and jobs are served in first-come-first-served order. The multi-server job model opens up new scaling regimes where both the number of servers that a job needs and the system load scale with the total number of servers. Within these scaling regimes, we derive the first results on stability, queueing probability, and the transient analysis of the number of jobs in the system for each class. In particular we derive sufficient conditions for zero queueing. Our analysis introduces a novel way of extracting information from the Lyapunov drift, which can be applicable to a broader scope of problems in queueing systems.

Large-System Insensitivity of Zero-Waiting Load Balancing Algorithms

“…ISSP, like the state-space collapse in the heavy-traffic analysis, is a general technique that may be used to study other complex stochastic systems, e.g. large-system insensitivity of load balancing algorithms for other models like those studied in [29,39,40,37,38].…”

“…Significant processes have been made over the past few years on understanding achieving asymptotic zero-waiting (as the system size approaches infinity) in a large-scale data center with distributed queues, including the classic supermarket model [14,8,32,17,3,4,30,24,25,23,22,45,9], models with data locality [40,31] and models where each job consists of parallel tasks [39,37,19], etc.…”

“…However, almost all these results assume exponential service time distributions. While each of these results [14,8,32,17,29,30,24,25,23,22,40,31,39,37,19,45,9] provided important insights of achieving zero-waiting in a practical system, theoretically, it is not clear whether these principles hold for general service times. This is a very important question to answer because it is well-known that service time distributions in real-world systems are not exponential.…”

Sharp Waiting-Time Bounds for Multiserver Jobs

“…A recent advance in understanding the delay of multiserver jobs is a characterization of the queueing probability in a large system by Wang et al [42], where the queueing probability is the probability that an arriving job has to queue rather than entering service immediately. Specifically, Wang et al [42] consider a multiserver job system with 𝑛 servers, and study the asymptotic scaling regime where 𝑛 becomes large. The scaling regime also allows server needs and arrival rates of jobs to scale with 𝑛 to capture the trend that the server needs in practice can be large and heterogeneous.…”

WCFS: A new framework for analyzing multiserver systems

“…Exact Markovchain methods suffer from the curse of dimensionality as the system grows large [18,29,44]. Asymptotic methods such as fluid and diffusion limits are often only applicable at high load, many servers, or both [45,47,50,51]. Lindley-type recursions can only be applied when the job completion process has a specific structure, renewing after every arrival [22,25].…”