Efficient Redundancy Techniques for Latency Reduction in Cloud Systems

Abstract

In computing-as-a-service frameworks, the computing cost is proportional to money spent on renting machines to run a job on the cloud 2 . PREVIOUS WORK AND MAIN CONTRIBUTIONS Related Previous WorkSystems Work: The use of redundancy to reduce latency is not new. One of the earliest instances is the use of multiple routing paths [Maxemchuk 1975] to send packets in networks; see [Kabatiansky et al. 2005, Chapter 7] for a detailed survey of other related work. A similar idea has been studied [Vulimiri et al. 2013] in the context of DNS queries. In large-scale cloud computing frameworks, several recent works in systems [Dean and Ghemawat 2008;Ananthanarayanan et al. 2013;Ousterhout et al. 2013] explore straggler mitigation techniques where redundant replicas of straggling tasks are launched to reduce latency. Although the use of redundancy has been explored in systems literature, there is little work on the rigorous analysis of how it affects latency, and in particular the cost of resources. Next we review some of that work.Exponential Service Time: The (n, k) fork-join system was first proposed in [Joshi et al. 2012;Joshi et al. 2014] to analyze content download latency from erasure coded distributed storage. These works consider that a content file coded into n chunks can be recovered by accessing any k out of the n chunks, where the service time X of each chunk is exponential. Even with the exponential assumption analyzing the (n, k) fork-join system is a hard problem. It is a generalization of the (n, n) fork-join system, which was actively studied in queueing literature [Flatto and Hahn 1984;Nelson and Tantawi 1988;Varki et al. 2008] around two decades ago.Recently, an analysis of latency with heterogeneous job classes for the replicated (k = 1) case with distributed queues is presented in [Gardner et al. 2015]. Other related works include [Shah et al. 2014;Kumar et al. 2014;Xiang et al. 2014;Kadhe et al. 2015]. A common thread in all these works is that they also assume exponential service time.General Service Time: Few practical systems have exponentially distributed service time. For example, studies of download time traces from Amazon S3 Chen et al. 2014] indicate that the service time is not exponential in practice, but instead a shifted exponential. For service time distributions that are 'new-worse-than-used' [Cao and Wang 1991], it is shown in [Koole and Righter 2008] that it is optimal to replicate a job at all servers in the system. The choice of scheduling policy for new-worse-than-used (NWU) and new-better-than-used (NBU) distributions is studied in [Kim et al. 2009;Shah et al. 2013;Sun et al. 2015]. The NBU and NWU notions are closely related to the log-concavity of service time studied in this work.The Cost of Redundancy: If we assume exponential service time then redundancy does not cause any increase in cost of server time. But since this is not true in practice, it is important to determine the cost of using redundancy. Simulation results with non-zero fixed cost of removal of redundant requests...

Rateless Codes for Near-Perfect Load Balancing in Distributed Matrix-Vector Multiplication

“…For the MDS and replication strategies, we reduce the queueing system to a fork-join queueing system with redundancy, and then use previous results [32,35] to obtain bounds on the mean response time. The results are presented in Appendix D.…”

Redundancy Scheduling in Systems with Bi-Modal Job Service Time Distributions

“…Queuing systems with redundancy, on the other hand, are studied in literature, e.g. [3], [11], [21], [25]. With redundancy, two scenarios for the cancellation of redundant copies of a jobs have been studied; cancellation after the first copy starts service, [3], [11], and cancellation after the first copy finishes service, [21], [25].…”

“…Two scenarios have been proposed for treating redundancy in distributed systems. In the first scenario, [3] and [11], copies of an arriving job are submitted to multiple servers and the redundant copies get cancelled once the first copy starts service. The first copy of a job starting the service is the one which faces the least-work-left queue among all the copies.…”

Service Rate Region of Content Access from Erasure Coded Storage

“…If, at demand λ, there exists a splitting strategy under which no storage system node receives requests at a rate in excess of its service rate, then λ is said to be in the achievable service rate region of the storage system. More formally, the storage system's achievable service rate region S is the set of all λ ∈ R K ≥0 such that there exists a splitting strategy with (4) For any λ = (λ1, . .…”