An Analysis of Load Imbalance in Scale-out Data Serving

NovakovićStanko,; DaglisAlexandros,; BugnionEdouard,; FalsafiBabak,; GrotBoris,

doi:10.1145/2964791.2901501

Cited by 4 publications

(10 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Prior research characterizing data access patterns in real-world settings has shown that popularity of individual items in a dataset often follows a power-law distribution [5,6,21,37,39,44]. In such a distribution, a small number of hot items receives a disproportionately high share of accesses, while the majority of the dataset observes relatively low access frequency.…”

Section: Motivation 21 Skew and Load Imbalancementioning

confidence: 99%

“…An important implication of popularity skew is the resulting load imbalance across the set of servers maintaining the dataset. As shown in Figure 2a, the server(s) responsible for the hottest keys may experience several times more load than an average server storing a slice of the dataset [37]. For instance, Figure 1 shows an example deployment of 128 servers and a data-serving workload with an access skew of α = 0.99.…”

Section: Motivation 21 Skew and Load Imbalancementioning

confidence: 99%

“…• Compared to a pure NUMA abstraction (e.g., FaRM [14], RackOut [37], FaSST [24]), adding a cache to each node can significantly lower the incidence of remote accesses. As Figure 3 shows, for a zipfian skew with an exponent a = 0.99 and a cache storing as little as 0.1% of the hottest data, 65% of requests will hit in the cache.…”

Section: Scale-out Ccnumamentioning

confidence: 99%

“…This is because the popularity distribution of objects is highly skewed, typically following power law distributions [5,21,37,43]. In other words, in the presence of skew, the server(s) serving the most popular objects will become saturated, thus becoming a bottleneck and limiting the throughput of the entire KVS.…”

Section: Introductionmentioning

confidence: 99%

“…The first class of techniques [16,22,32] uses a dedicated cache for storing popular keys to filter the skew. The second class of techniques (FaRM [14] and RackOut [37]) mitigate skew by evenly distributing read requests across all servers of the KVS regardless of the object's location. For low latency, the servers use an RDMA-enabled interconnect to access objects that reside at other servers.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Scale-out ccNUMA

Gavrielatos

Katsarakis

Joshi

et al. 2018

Proceedings of the Thirteenth EuroSys Conference

Self Cite

View full text Add to dashboard Cite

Today's cloud based online services are underpinned by distributed key-value stores (KVS). Such KVS typically use a scale-out architecture, whereby the dataset is partitioned across a pool of servers, each holding a chunk of the dataset in memory and being responsible for serving queries against the chunk. One important performance bottleneck that a KVS design must address is the load imbalance caused by skewed popularity distributions. Despite recent work on skew mitigation, existing approaches offer only limited benefit for high-throughput in-memory KVS deployments.In this paper, we embrace popularity skew as a performance opportunity. Our insight is that aggressively caching popular items at all nodes of the KVS enables both load balance and high throughput -a combination that has eluded previous approaches. We introduce symmetric caching, wherein every server node is provisioned with a small cache that maintains the most popular objects in the dataset. To ensure consistency across the caches, we use high-throughput fully-distributed consistency protocols. A key result of this work is that strong consistency guarantees (per-key linearizability) need not compromise on performance. In a 9-node RDMA-based rack and with modest write ratios, our prototype design, dubbed ccKVS, achieves 2.2× the throughput of the state-ofthe-art KVS while guaranteeing strong consistency.

show abstract

Section: Motivation 21 Skew and Load Imbalancementioning

confidence: 99%

Section: Motivation 21 Skew and Load Imbalancementioning

confidence: 99%

Section: Scale-out Ccnumamentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Scale-out ccNUMA

Gavrielatos

Katsarakis

Joshi

et al. 2018

Proceedings of the Thirteenth EuroSys Conference

Self Cite

View full text Add to dashboard Cite

show abstract

Odyssey

Gavrielatos

Katsarakis

Nagarajan

2021

Proceedings of the Sixteenth European Conference on Computer Systems

View full text Add to dashboard Cite

Get/Put Key-Value Stores (KVSes) rely on replication protocols to enforce consistency and guarantee availability. Today's modern hardware, with manycore servers and RDMAcapable networks, challenges the conventional wisdom on protocol design. In this paper, we investigate the impact of modern hardware on the performance of strongly-consistent replication protocols.First, we create an informal taxonomy of replication protocols, based on which we carefully select 10 protocols for analysis. Secondly, we present Odyssey, a framework tailored towards protocol implementation for multi-threaded, RDMA-enabled, in-memory, replicated KVSes. We implement all 10 protocols over Odyssey, and perform the first apples-to-apples comparison of replication protocols over modern hardware.Our comparison characterizes the protocol design space, revealing the performance capabilities of different classes of protocols on modern hardware. Among other things, our results demonstrate that some of the protocols that were efficient in yesterday's hardware are not so today because they cannot take advantage of the abundant parallelism and fast networking present in modern hardware. Conversely, some protocols that were inefficient in yesterday's hardware are very attractive today. We distill our findings in a concise set of general guidelines and recommendations for protocol selection and design in the era of modern hardware.

show abstract

Balancing Distributed Key-Value Stores with Efficient In-Network Redirecting

Shi¹,

Fei²,

Wen³

et al. 2019

Electronics

View full text Add to dashboard Cite

Today’s cloud-based online services are underpinned by distributed key-value stores (KVSs). Keys and values are distributed across back-end servers in such scale-out systems. One primary real-life performance bottleneck occurs when storage servers suffer from load imbalance under skewed workloads. In this paper, we present KVSwitch, a centralized self-managing load balancer that leverages the power and flexibility of emerging programmable switches. The balance is achieved by dynamically predicting the hot items and by creating replication strategies according to KVS loading. To overcome the challenges in realizing KVSwitch given the limitations of the switch hardware, we decompose KVSwitch’s functions and carefully design them for the heterogeneous processors inside the switch. We prototype KVSwitch in a Tofino switch. Experimental results show that our solution can effectively keep the KVS servers balanced even under highly skewed workloads. Furthermore, KVSwitch only replicates 70 % of hot items and consumes 9.88 % of server memory rather than simply replicating all hot items to each server.

show abstract

An Analysis of Load Imbalance in Scale-out Data Serving

Cited by 4 publications

References 6 publications

Scale-out ccNUMA

Scale-out ccNUMA

Odyssey

Balancing Distributed Key-Value Stores with Efficient In-Network Redirecting

Contact Info

Product

Resources

About