Characterizing Load Imbalance in Real-World Networked Caches

Abstract

Modern Web services rely extensively upon a tier of in-memory caches to reduce request latencies and alleviate load on backend servers. Within a given cache, items are typically partitioned across cache servers via consistent hashing, with the goal of balancing the number of items maintained by each cache server. Effects of consistent hashing vary by associated hashing function and partitioning ratio. Most real-world workloads are also skewed, with some items significantly more popular than others. Inefficiency in addressing both issues can create an imbalance in cache-server loads.We analyze the degree of observed load imbalance, focusing on read-only traffic against Facebook's graph cache tier in Tao. We investigate the principal causes of load imbalance, including data co-location, non-ideal hashing scenarios, and hot-spot temporal effects. We also employ trace-drive analytics to study the benefits and limitations of current loadbalancing methods, suggesting areas for future research.

An Analysis of Load Imbalance in Scale-out Data Serving

“…Service providers are well aware of the problems arising from skew-induced load imbalance [3,6]: servers that hold the most popular micro-shards can quickly become overwhelmed with user requests, degrading the whole system's performance and service quality. Data replication is a widely-used technique for dealing with such load imbalance in the datacenter.…”

“…The hottest micro-shard alone accounts for 4.2% of the total traffic, which is 20× the average load of an entire server in a 512-node cluster, and 85× in a 2048-node cluster of the same aggregate capacity. For a read-only workload, one solution is to combine aggressive replication across the cluster with caching at the application tier [5,6]. Unfortunately, neither replication nor caching is free for read-write workloads, as each copy needs to be updated on each write, regardless of the consistency model.…”

The Case for RackOut

“…In such a setting, skewed distributions emerge naturally, as data popularity varies greatly. Previous work has shown that data popularity distributions in real-world KVS workloads follow a power-law distribution [9,11,27,32], resulting in an access frequency imbalance, commonly referred to as skew. This skewed distribution is accurately represented by the powerlaw Zipfian distribution [9,15,17,27,37,55].…”

“…Service providers are well aware of the problems that arise from skew-induced load imbalance [20,21,32]: servers holding the most popular micro-shards can quickly become overwhelmed with user requests, degrading the whole system's performance and service quality. Data replication is a widely used technique that deals with such load imbalance in the datacenter.…”

“…Read requests are loadbalanced among all the servers within each RackOut unit, while write requests are always directed to the server holding the micro-shard. Even though data is never replicated within a RackOut unit, RackOut is compatible and synergistic with dynamic replication [11,21,31,32]. Since the RackOut technique already balances requests within a RackOut unit (using one-sided operations rather than replication), replication is only required across RackOut units, which reduces the overheads associated with load monitoring, creating replicas, and ongoing consistency maintenance.…”