Ubik

Kasture, Harshad; Sánchez, Daniel

doi:10.1145/2541940.2541944

Cited by 114 publications

(4 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Improving server cost-efficiency via multi-tenancy has been studied extensively in prior literature [18], [19], [20], [21], [22], [23]. As co-located multi-tenant tasks contend for shared resources, prior work has focused on how to minimize interference and performance unpredictability.…”

Section: Related Workmentioning

confidence: 99%

Hera: A Heterogeneity-Aware Multi-Tenant Inference Server for Personalized Recommendations

Choi¹,

Kim²,

Rhu³

2023

Preprint

View full text Add to dashboard Cite

While providing low latency is a fundamental requirement in deploying recommendation services, achieving high resource utility is also crucial in cost-effectively maintaining the datacenter. Co-locating multiple workers of a model is an effective way to maximize query-level parallelism and server throughput, but the interference caused by concurrent workers at shared resources can prevent server queries from meeting its SLA. Hera utilizes the heterogeneous memory requirement of multi-tenant recommendation models to intelligently determine a productive set of co-located models and its resource allocation, providing fast response time while achieving high throughput. We show that Hera achieves an average 37.3% improvement in effective machine utilization, enabling 26% reduction in required servers, significantly improving upon the baseline recommedation inference server.

show abstract

Section: Related Workmentioning

confidence: 99%

Hera: A Heterogeneity-Aware Multi-Tenant Inference Server for Personalized Recommendations

Choi¹,

Kim²,

Rhu³

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…S 2 C 2 differs from these techniques because it is a pro-active approach to straggler mitigation. In works [18,25,46] the authors explore system sources of tail latency from system and implement mechanisms to eliminate these causes. In another set of works, [9,24,31,48], authors focus on improving resource efficiency while providing low latency.…”

Section: Related Workmentioning

confidence: 99%

Slack squeeze coded computing for adaptive straggler mitigation

Narra

Lin

Kiamari

et al. 2019

Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

While performing distributed computations in today's cloud-based platforms, execution speed variations among compute nodes can significantly reduce the performance and create bottlenecks like stragglers. Coded computation techniques leverage coding theory to inject computational redundancy and mitigate stragglers in distributed computations. In this paper, we propose a dynamic workload distribution strategy for coded computation called Slack Squeeze Coded Computation (S 2 C 2 ). S 2 C 2 squeezes the compute slack (i.e., overhead) that is built into the coded computing frameworks by efficiently assigning work for all fast and slow nodes according to their speeds and without needing to re-distribute data. We implement an LSTM-based speed prediction algorithm to predict speeds of compute nodes. We evaluate S 2 C 2 on linear algebraic algorithms, gradient descent, graph ranking, and graph filtering algorithms. We demonstrate 19% to 39% reduction in total computation latency using S 2 C 2 compared to job replication and coded computation. We further show how S 2 C 2 can be applied beyond matrix-vector multiplication.

show abstract

“…They propose a QoS metric that modulates the allocated cache space for a given thread. Ubik [16] employs ZCaches [17], a radical change from traditional caches, to partition the cache. Their goal is to provide QoS while at the same time improving the performance of batch applications.…”

Section: Related Workmentioning

confidence: 99%

A Hardware Approach to Fairly Balance the Inter-Thread Interference in Shared Caches

Selfa

Sahuquillo

Petit

et al. 2017

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

Shared caches have become the common design choice in the vast majority of modern multi-core and many-core processors, since cache sharing improves throughput for a given silicon area. Sharing the cache, however, has a downside: the requests from multiple applications compete among them for cache resources, so the execution time of each application increases over isolated execution. The degree in which the performance of each application is affected by the interference becomes unpredictable yielding the system to unfairness situations. This paper proposes Fair-Progress Cache Partitioning (FPCP), a low-overhead hardware-based cache partitioning approach that addresses system fairness. FPCP reduces the interference by allocating to each application a cache partition and adjusting the partition sizes at runtime. To adjust partitions, our approach estimates during multicore execution the time each application would have taken in isolation, which is challenging. The proposed approach has two main differences over existing approaches. First, FPCP distributes cache ways incrementally, which makes the proposal less prone to estimation errors. Second, the proposed algorithm is much less costly than the state-of-the-art ASM-Cache approach. Experimental results show that, compared to ASM-Cache, FPCP reduces unfairness by 48% in four-application workloads and by 28% in eight-application workloads, without harming the performance.

show abstract

Ubik

Cited by 114 publications

References 62 publications

Hera: A Heterogeneity-Aware Multi-Tenant Inference Server for Personalized Recommendations

Hera: A Heterogeneity-Aware Multi-Tenant Inference Server for Personalized Recommendations

Slack squeeze coded computing for adaptive straggler mitigation

A Hardware Approach to Fairly Balance the Inter-Thread Interference in Shared Caches

Contact Info

Product

Resources

About