DIAL: Reducing Tail Latencies for Cloud Applications via Dynamic Interference-aware Load Balancing

Javadi, Seyyed Ahmad; Gandhi, Anshul

doi:10.1109/icac.2017.17

Cited by 35 publications

(11 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…V. RELATED WORK Building distributed systems that offer guarantees on their timely execution while the system is subject to uncertainty and changes is a challenging task. Bounding latencies is of utmost importance, but this is quite difficult in the presence of changes [5,6,14,24,25,42]. Changes are unpredictable, they can be dramatic, and they can include malfunctioning [23], slow down [12], failures [18], and much more.…”

Section: B Experimental Resultsmentioning

confidence: 99%

“…To determine the size of data centers, and properly dimension the resources to be allocated in each geographic location, most data center owners use predictions of the computational needs [29,36]. The computational resource within a data center is then used to serve requests coming from multiple clients, providing the illusion of infinite capacity and, as a result, the possibility of bounding the latency [5,6,14,24,25,42]. To do so the architecture uses multiple instances of the same application, here called replicas, and predictions and estimations of traffic and needed computational capacity.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Cloud Application Predictability through Integrated Load-Balancing and Service Time Control

Nylander

Andrén

Årzén

et al. 2018

2018 IEEE International Conference on Autonomic Computing (ICAC)

View full text Add to dashboard Cite

Cloud computing provides the illusion of infinite capacity to application developers. However, data center provisioning is complex and it is still necessary to handle the risk of capacity shortages. To handle capacity shortages, graceful degradation techniques sacrifice user experience for predictability. In all these cases, the decision making policy that determines the degradation interferes with other decisions happening at the infrastructure level, like load-balancing choices. Here, we reconcile the two approaches, developing a load-balancing strategy that also handles capacity shortages and graceful degradation when necessary. The proposal is based on a sound control-theoretical approach. The design of the approach avoids the pitfalls of interfering control decisions. We describe the technique and provide evidence that it allows us to achieve higher performance in terms of emergency management and user experience.

show abstract

Section: B Experimental Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Cloud Application Predictability through Integrated Load-Balancing and Service Time Control

Nylander

Andrén

Årzén

et al. 2018

2018 IEEE International Conference on Autonomic Computing (ICAC)

View full text Add to dashboard Cite

show abstract

“…In DIAL [19], interference detection is accomplished using decision tree-based classifier to find the dominant source of resource contention. To quantify the resource interference impact on a webserver application's tail response, a queuing model is utilized to determine the application's response time under contention.…”

Section: R a F Tmentioning

confidence: 99%

FECBench: A Holistic Interference-aware Approach for Application Performance Modeling

Barve

Shekhar

Chhokra

et al. 2019

2019 IEEE International Conference on Cloud Engineering (IC2E)

View full text Add to dashboard Cite

Services hosted in multi-tenant cloud platforms often encounter performance interference due to contention for non-partitionable resources, which in turn causes unpredictable behavior and degradation in application performance. To grapple with these problems and to define effective resource management solutions for their services, providers often must expend significant efforts and incur prohibitive costs in developing performance models of their services under a variety of interference scenarios on different hardware. This is a hard problem due to the wide range of possible co-located services and their workloads, and the growing heterogeneity in the runtime platforms including the use of fog and edge-based resources, not to mention the accidental complexity in performing application profiling under a variety of scenarios. To address these challenges, we present FECBench (Fog/Edge/Cloud Benchmarking), an open source framework comprising a set of 106 applications covering a wide range of application classes to guide providers in building performance interference prediction models for their services without incurring undue costs and efforts. Through the design of FECBench, we make the following contributions. First, we develop a technique to build resource stressors that can stress multiple system resources all at once in a controlled manner, which helps to gain insights into the impact of interference on an application's performance. Second, to overcome the need for exhaustive application profiling, FECBench intelligently uses the design of experiments (DoE) approach to enable users to build surrogate performance models of their services. Third, FECBench maintains an extensible knowledge base of application combinations that create resource stresses across the multidimensional resource design space. Empirical results using realworld scenarios to validate the efficacy of FECBench show that the predicted application performance has a median error of only 7.6% across all test cases, with 5.4% in the best case and 13.5% in the worst case.

show abstract

Section: Introductionmentioning

confidence: 99%

“…The load-balancing approach we suggest is designed specifically for DSAG, but is inspired by the large number of previous works on the topic; see, e.g., [35], [36], [37], [38], and references therein. These suggest approaches to balance either i) the complexity of the subtasks that make up a particular large computation (e.g., [35], [36]), or ii) incoming requests between instances of a distributed application, such as a web server (e.g., [37], [38]). The approach we suggest, like those of [37], [38], but unlike [35], [36], accounts for latency differences between servers and over time-as is the case in the cloud-but balances the complexity of subtasks, as in [35], [36].…”

Section: Introductionmentioning

confidence: 99%

DSAG: A mixed synchronous-asynchronous iterative method for straggler-resilient learning

Severinson¹,

Rosnes²,

Rouayheb³

et al. 2021

Preprint

View full text Add to dashboard Cite

We consider straggler-resilient learning. In many previous works, e.g., in the coded computing literature, straggling is modeled as random delays that are independent and identically distributed between workers. However, in many practical scenarios, a given worker may straggle over an extended period of time. We propose a latency model that captures this behavior and is substantiated by traces collected on Microsoft Azure, Amazon Web Services (AWS), and a small local cluster. Building on this model, we propose DSAG, a mixed synchronous-asynchronous iterative optimization method, based on the stochastic average gradient (SAG) method, that combines timely and stale results. We also propose a dynamic load-balancing strategy to further reduce the impact of straggling workers. We evaluate DSAG for principal component analysis, cast as a finite-sum optimization problem, of a large genomics dataset, and for logistic regression on a cluster composed of 100 workers on AWS, and find that DSAG is up to about 50% faster than SAG, and more than twice as fast as coded computing methods, for the particular scenario that we consider.

show abstract

DIAL: Reducing Tail Latencies for Cloud Applications via Dynamic Interference-aware Load Balancing

Cited by 35 publications

References 26 publications

Cloud Application Predictability through Integrated Load-Balancing and Service Time Control

Cloud Application Predictability through Integrated Load-Balancing and Service Time Control

FECBench: A Holistic Interference-aware Approach for Application Performance Modeling

DSAG: A mixed synchronous-asynchronous iterative method for straggler-resilient learning

Contact Info

Product

Resources

About