Scale-out ccNUMA

Abstract

Today's cloud based online services are underpinned by distributed key-value stores (KVS). Such KVS typically use a scale-out architecture, whereby the dataset is partitioned across a pool of servers, each holding a chunk of the dataset in memory and being responsible for serving queries against the chunk. One important performance bottleneck that a KVS design must address is the load imbalance caused by skewed popularity distributions. Despite recent work on skew mitigation, existing approaches offer only limited benefit for high-throughput in-memory KVS deployments.In this paper, we embrace popularity skew as a performance opportunity. Our insight is that aggressively caching popular items at all nodes of the KVS enables both load balance and high throughput -a combination that has eluded previous approaches. We introduce symmetric caching, wherein every server node is provisioned with a small cache that maintains the most popular objects in the dataset. To ensure consistency across the caches, we use high-throughput fully-distributed consistency protocols. A key result of this work is that strong consistency guarantees (per-key linearizability) need not compromise on performance. In a 9-node RDMA-based rack and with modest write ratios, our prototype design, dubbed ccKVS, achieves 2.2× the throughput of the state-ofthe-art KVS while guaranteeing strong consistency.

Mitigating Load Imbalance in Distributed Data Serving with Rack-Scale Memory Pooling

“…Since our model is highly optimistic, maintaining hundreds of replicas would in reality introduce much more overhead and lead to faster saturation. For 1% of writes and less, replication introduces smaller performance penalty [32]. As Figure 11 shows, a combination of moderate replication with RackOut represents an effective design point; for example, GF=16 with only a few replicas matches the utilization achieved by scale-out with tens of replicas.…”

Kite

“…Every node in Kite maintains a local KVS. The implementation of the KVS is largely based on MICA [54] as found in [40], with the addition of sequence locks (seqlocks) [45], from [24], to enable multi-threading. Adapting MICA for ES and ABD.…”

Balancing Distributed Key-Value Stores with Efficient In-Network Redirecting

“…Zipf has a skewness parameter β, a higher value of which denotes a more acute imbalance. The most common value for β in the literature is 0.99 [5,8,9,11]; this value is also used in evaluations for KVSwitch (Section 5).…”

“…Skewness in workload results in load imbalance across storage servers. With β = 0.99 for 10,000 items, the hottest 100 items account for about 51.8% of all queries and the busiest server receives 7× the average load [11]. As shown in Figure 1a, the server responsible for hot item Z experiences far greater loads than the server containing cold item D. A similar problem is also observed in the industry; in KVS for Alibaba, the world's largest retailer, a few servers will crash under the weight of numerous queries while many other servers are almost idle with no mitigation techniques [3].…”

“…A number of techniques have been proposed to mitigate this well-established skew problem. These techniques can be broadly divided into two classes: the first class [8][9][10] uses a dedicated cache to store hot items and to absorb the associated queries, while the second [3,11,12] replicates hot items across multiple servers and redirects queries to these servers to avoid imbalance.…”