Optimizing Locality-Aware Memory Management of Key-Value Caches

Hu, Xiameng; Wang, Xiaolin; Zhou, Licheng; Luo, Yingwei; Ding, Chen; Jiang, Song; Wang, Zhenlin

doi:10.1109/tc.2016.2618920

Cited by 27 publications

(44 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The correctness has been evaluated and validated for solo-use cache, including the two initial studies of the footprint theory [15], [16]. Independent validation can be found in the use of the footprint theory in optimal program symbiosis in shared cache [12], optimal memory allocation in Memcached [2], and a study from the OS community on server cache performance for disk access traces [13].…”

Section: Discussion On Optimalitymentioning

confidence: 99%

“…Wang et al tested the analysis on program execution traces for CPU cache [12], Hu et al on keyvalue access traces for Memcached [2], and Wires et al on disk access traces for server cache [13]. The three studies re-implemented the footprint analysis independently and reported high accuracy through extensive testing.…”

Section: Related Workmentioning

confidence: 99%

“…The three studies re-implemented the footprint analysis independently and reported high accuracy through extensive testing. Hu et al tested the speed of convergence, i.e., how quickly the memory allocation stablizes under a steadystate workload, and found that optimal partition converges 4 times faster than free-for-all sharing [2]. Finally, Wang et al showed strong correlation (coefficient 0.938) between the predicted miss ratio and measured co-run speed [12].…”

Section: Related Workmentioning

confidence: 99%

“…His home institution is Huazhong University of Science and Technology, Wuhan, China. 2 Parihar et al presented a counter-based hardware mechanism to provide programs the protection of partitioning, without the risk of unused space [4]. and free-for-all sharing can be seen as opposite edge cases of partition-sharing.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Optimal Cache Partition-Sharing

Brock

Ding

et al. 2015

2015 44th International Conference on Parallel Processing

Self Cite

View full text Add to dashboard Cite

When a cache is shared by multiple cores, its space may be allocated either by sharing, partitioning, or both. We call the last case partition-sharing. This paper studies partition-sharing as a general solution, and presents a theory an technique for optimizing partition-sharing. We present a theory and a technique to optimize partition sharing. The theory shows that the problem of partition-sharing is reducible to the problem of partitioning. The technique uses dynamic programming to optimize partitioning for overall miss ratio, and for two different kinds of fairness.Finally, the paper evaluates the effect of optimal cache sharing and compares it with conventional solutions for thousands of 4-program co-run groups, with nearly 180 million different ways to share the cache by each co-run group. Optimal partition-sharing is on average 26% better than freefor-all sharing, and 98% better than equal partitioning. We also demonstrate the trade-off between optimal partitioning and fair partitioning.

show abstract

Section: Discussion On Optimalitymentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Optimal Cache Partition-Sharing

Brock

Ding

et al. 2015

2015 44th International Conference on Parallel Processing

Self Cite

View full text Add to dashboard Cite

show abstract

“…More recently, Hu et al propose to use miss ratio curve for quantifying access locality and use the curve to determine the optimal space allocation for each class [9]. The optimality can be defined in terms of either hit ratio or average request service time.…”

Section: Background and Related Workmentioning

confidence: 99%

A Penalty Aware Memory Allocation Scheme for Key-Value Cache

Patton

Moore

et al. 2015

2015 44th International Conference on Parallel Processing

Self Cite

View full text Add to dashboard Cite

Key-value caches, represented by Memcached, play a critical role in data centers. Its efficacy can significantly impact users' perceived service time and back-end systems' workloads. A central issue in the in-memory cache's management is memory allocation, or how the limited space is distributed for storing keyvalue items of various sizes. When a cache is full, the allocation issue is how to conduct replacement operations on items of different sizes. To effectively address the issue, a practitioner must simultaneously consider three factors, which are access locality, item size, and miss penalty. Existing designs consider only one or two of the first two factors, and pay little attention on miss penalty. This inadequacy can substantially compromise utilization of cache space and request service time.In this paper we propose a Penalty Aware Memory Allocation scheme (PAMA) that takes all three factors into account. While the three different factors cannot be directly compared to each other in a quantitative manner, PAMA uses their impacts on service time to determine where a unit of memory space should be (de)allocated. The impacts are quantified as the decrease (or increase) of service time if a unit of space is allocated (or deallocated). PAMA efficiently tracks access pattern and use of memory, and speculatively evaluates the impacts to enable penalty-aware memory allocation for KV caches. Our evaluation with real-world Memcached workload traces demonstrates that PAMA can significantly reduce request service time compared to other representative KV cache management schemes.

show abstract