Reducing Latency via Redundant Requests

Abstract

Recent computer systems research has proposed using redundant requests to reduce latency. The idea is to run a request on multiple servers and wait for the first completion (discarding all remaining copies of the request). However there is no exact analysis of systems with redundancy. This paper presents the first exact analysis of systems with redundancy. We allow for any number of classes of redundant requests, any number of classes of non-redundant requests, any degree of redundancy, and any number of heterogeneous servers. In all cases we derive the limiting distribution on the state of the system. In small (two or three server) systems, we derive simple forms for the distribution of response time of both the redundant classes and non-redundant classes, and we quantify the "gain" to redundant classes and "pain" to non-redundant classes caused by redundancy. We find some surprising results. First, the response time of a fully redundant class follows a simple Exponential distribution and that of the non-redundant class follows a Generalized Hyperexponential. Second, fully redundant classes are "immune" to any pain caused by other classes becoming redundant. We also compare redundancy with other approaches for reducing latency, such as optimal probabilistic splitting of a class among servers (Opt-Split) and Join-the-Shortest-Queue (JSQ) routing of a class. We find that, in many cases, redundancy outperforms JSQ and Opt-Split with respect to overall response time, making it an attractive solution.

Mean-field-analysis of coding versus replication in cloud storage systems

“…In [6], the authors provided delay bounds under the MDS code. Papers (e.g., [4], [7]- [13]) studied the delay performance of redundant requests in various settings. To the best of our knowledge, none of these works are able to characterize or analytically bound the performance improvement under coding compared with replication.…”

Cutting Latency Tail: Analyzing and Validating Replication without Canceling

“…In fact, canceling plays a key role in obtaining the scarce analytical solutions available for request replication [10], [11]. For instance, canceling enables [11] to reduce the problem to a single M/G/1 queue in the early-canceling case or to an approximate M/G/C queue in the late-canceling one.…”

“…For instance, canceling enables [11] to reduce the problem to a single M/G/1 queue in the early-canceling case or to an approximate M/G/C queue in the late-canceling one. Similarly, canceling is key in obtaining the solution to the Markov chain model in [10]. The absence of canceling allows servers to evolve asynchronously, requiring a more complex analysis.…”

“…However, in other applications, processing times depend on the intrinsic details of the request, which impacts all of its replicas. In such cases correlation among the replicas of a request becomes significant; whilst commonly ignored [6]- [8], [10], [14], it has only recently been taken into account in [11], [20]. In contrast, this paper considers cases of both independent and correlated processing times among replicas of the same request.…”

Simplex Queues for Hot-Data Download

“…. $15.00 DOI: 10.1145/nnnnnnn.nnnnnnn simultaneously download from di erent replicas, or each user can issue simultaneous requests to multiple replicas and wait for the rst download to nish [6,7,15]. Data access systems arising in erasure coded storage have received a lot of a ention for all-data (see for example [12,14,19,24] and references therein) and some for hot-data download [16,17], and usually involve single or multiple inter-dependent fork-join queueing systems.…”