On the Power of (Even a Little) Resource Pooling

Tsitsiklis, John N.; Xu, Kuang

doi:10.1287/11-ssy033

Cited by 95 publications

(98 citation statements)

References 25 publications

(31 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The design and performance of flexible server systems has been studied in [6,[31][32][33]. In a flexible server system, traditionally, when a server becomes available it chooses the queue from which to take its next job according to some policy.…”

Section: Flexible Server Systemsmentioning

confidence: 99%

Queueing with redundant requests: exact analysis

et al. 2016

View full text Add to dashboard Cite

Recent computer systems research has proposed using redundant requests to reduce latency. The idea is to run a request on multiple servers and wait for the first completion (discarding all remaining copies of the request). However, there is no exact analysis of systems with redundancy. This paper presents the first exact analysis of systems with redundancy. We allow for any number of classes of redundant requests, any number of classes of non-redundant requests, any degree of redundancy, and any number of heterogeneous servers. In all cases we derive the limiting distribution of the state of the system. In small (two or three server) systems, we derive simple forms for the distribution of response time of both the redundant classes and non-redundant classes, and we quantify the "gain" to redundant classes and "pain" to non-redundant classes caused by redundancy. We find some surprising results. First, the response time of a fully redundant class follows a simple exponential distribution and that of the non-redundant class follows a generalized hyperexponential. Second, fully redundant classes are "immune" to any pain caused by other classes becoming redundant. We also compare redundancy with other approaches for reducing latency, such as optimal probabilistic splitting of a class among servers (Opt-Split) and join-the-shortest-queue (JSQ) routing of a class. We find that, in many cases, redundancy outperforms JSQ and Opt-Split with respect to overall response time, making it an attractive solution.

show abstract

Section: Flexible Server Systemsmentioning

confidence: 99%

Queueing with redundant requests: exact analysis

et al. 2016

View full text Add to dashboard Cite

show abstract

“…The papers by Alanyali and Dashouk [1] and Tsitsiklis and Xu [14] are closely related to this paper. Both consider scheduling in the presence of a large number of buffers.…”

Section: Introductionmentioning

confidence: 78%

“…Such mean-field quantities have been used in analyzing various scheduling and load balancing policies; see, for example, [1], [13], [14]. However, under the randomized LQF policy, we can expect from [1] that, whenever lim n→∞ d(n) = ∞,…”

Section: Model and Notationmentioning

confidence: 99%

“…A certain time scaling that is not present in [1] is essential for the validity of our limit theorems. In [14] the authors analyzed a hybrid system with centralized and distributed processing capacity in a setting similar to ours. Their work exposes a dramatic improvement in performance in the presence of centralization compared to a fully distributed system.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Randomized longest-queue-first scheduling for large-scale buffered systems

Dieker

Suk

2015

Adv. Appl. Probab.

View full text Add to dashboard Cite

We develop diffusion approximations for parallel-queueing systems with the randomized longest-queue-first scheduling (LQF) algorithm by establishing new mean-field limit theorems as the number of buffers n → ∞. We achieve this by allowing the number of sampled buffers d = d(n) to depend on the number of buffers n, which yields an asymptotic 'decoupling' of the queue length processes. We show through simulation experiments that the resulting approximation is accurate even for moderate values of n and d(n). To the best of the authors' knowledge, this is the first derivation of diffusion approximations for a queueing system in the large-buffer mean-field regime. Another noteworthy feature of our scaling idea is that the randomized LQF algorithm emulates the LQF algorithm, yet is computationally more attractive. The analysis of the system performance as a function of d(n) is facilitated by the multi-scale nature in our limit theorems: the various processes we study have different space scalings. This allows us to show the trade-off between performance and complexity of the randomized LQF scheduling algorithm.

show abstract

“…The design and performance of flexible server systems has been studied in [5,29,30,31]. In a flexible server system, traditionally, when a server becomes available, it chooses the queue from which to take its next job according to some policy.…”

Section: Flexible Server Systemsmentioning

confidence: 99%

Reducing Latency via Redundant Requests

Gardner

Zbarsky

Doroudi

et al. 2015

Proceedings of the 2015 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems

View full text Add to dashboard Cite

Recent computer systems research has proposed using redundant requests to reduce latency. The idea is to run a request on multiple servers and wait for the first completion (discarding all remaining copies of the request). However there is no exact analysis of systems with redundancy.This paper presents the first exact analysis of systems with redundancy. We allow for any number of classes of redundant requests, any number of classes of non-redundant requests, any degree of redundancy, and any number of heterogeneous servers. In all cases we derive the limiting distribution on the state of the system.In small (two or three server) systems, we derive simple forms for the distribution of response time of both the redundant classes and non-redundant classes, and we quantify the "gain" to redundant classes and "pain" to non-redundant classes caused by redundancy. We find some surprising results. First, the response time of a fully redundant class follows a simple Exponential distribution and that of the non-redundant class follows a Generalized Hyperexponential. Second, fully redundant classes are "immune" to any pain caused by other classes becoming redundant.We also compare redundancy with other approaches for reducing latency, such as optimal probabilistic splitting of a class among servers (Opt-Split) and Join-the-Shortest-Queue (JSQ) routing of a class. We find that, in many cases, redundancy outperforms JSQ and Opt-Split with respect to overall response time, making it an attractive solution.

show abstract

On the Power of (Even a Little) Resource Pooling

Cited by 95 publications

References 25 publications

Queueing with redundant requests: exact analysis

Queueing with redundant requests: exact analysis

Randomized longest-queue-first scheduling for large-scale buffered systems

Reducing Latency via Redundant Requests

Contact Info

Product

Resources

About