Large-scale in-memory object caches such as memcached are widely used to accelerate popular web sites and to reduce burden on backend databases. Yet current cache systems give cache operators limited information on what resources are required to optimally accommodate the present workload. This paper focuses on a key question for cache operators: how much total memory should be allocated to the in-memory cache tier to achieve desired performance?We present our MIMIR system: a lightweight online profiler that hooks into the replacement policy of each cache server and produces graphs of the overall cache hit rate as a function of memory size. The profiler enables cache operators to dynamically project the cost and performance impact from adding or removing memory resources within a distributed in-memory cache, allowing "what-if" questions about cache performance to be answered without laborious offline tuning. Internally, MIMIR uses a novel lock-free algorithm and lookup filters for quickly and dynamically estimating hit rate of LRU caches.Running MIMIR as a profiler requires minimal changes to the cache server, thanks to a lean API. Our experiments show that MIMIR produces dynamic hit rate curves with over 98% accuracy and 2 − 5% overhead on request latency and throughput when MIMIR is run in tandem with memcached, suggesting online cache profiling can be a practical tool for improving provisioning of large caches.
No abstract
In-memory object caches, such as memcached, are critical to the success of popular web sites, such as Facebook [3], by reducing database load and improving scalability [2]. The prominence of caches implies that configuring their ideal memory size has the potential for significant savings on computation resources and energy costs, but unfortunately cache configuration is poorly understood. The modern practice of manually tweaking live caching systems takes significant effort and may both increase the variance for client request latencies and impose high load on the database backend.Contributions. We provide an efficient online algorithm to estimate how an LRU cache would perform using a different memory allocation, continually exposing a hit rate curve as a function of space ( Figure 1). Our method is lock-free and compatible with modern multithreaded cache servers, such as memcached.Approach. For a cache of size n, the challenge is to generate a hit rate curve for cache sizes ranging from 0 to 2n. First, to predict how a cache would perform beyond the current allocation of n, we track metadata for n additional dataless elements, so-called "ghosts" [1]. While technically a cache miss, a hit on a ghost provides information about how the larger cache allocation would fare under the same workload [4].To track statistics, our method splits the LRU stack into a list of variably sized buckets. The first bucket represents the top of the LRU stack and the last bucket represents the tail. Whenever a cache hit occurs, the element e causing the hit is moved to the first bucket. The stack distance of e can then be estimated by summing up the number of elements in buckets in front of where e was 0 1000 2000 3000 Cache size (items) 0.0 0.1 0.2 0.3 0.4 0.5 0.6 Cumulative hit rate Current allocation LRU Figure 1: Hit rate curve. The normalized cumulative cache hit rate achieved for cache sizes different from the current allocation.in the list. Next, the hit rate curve is updated (Figure 2). The elements are then aged to maintain about the same number of elements per bucket. For aging, we can trade off performance (constant time vs. linear in number of buckets) for estimation accuracy. If accuracy is favored, a global average stack distance is maintained and only certain elements are aged.Results. The faster algorithm, ROUNDER, achieves over 96% accuracy measured by the mean average error of the hit rate curve on a wide variety of cache workloads, compared to over 99% for the slower one. We implemented and evaluated ROUNDER in memcached and found negligible throughput degradation on standard benchmarks. We conclude that online generation of hit rate curves are both useful for provisioning and monitoring, and can be made practical for large cache systems. Figure 2: The ROUNDER algorithm. Steps to update the hit rate curve and bucket lists of the LRU stack when item e is hit in the cache.
The growing pressure on cloud application scalability has accentuated storage performance as a critical bottleneck. Although cache replacement algorithms have been extensively studied, cache prefetching -reducing latency by retrieving items before they are actually requestedremains an underexplored area. Existing approaches to history-based prefetching, in particular, provide too few benefits for real systems for the resources they cost.We propose MITHRIL, a prefetching layer that efficiently exploits historical patterns in cache request associations. MITHRIL is inspired by sporadic association rule mining and only relies on the timestamps of requests. Through evaluation of 135 block-storage traces, we show that MITHRIL is effective, giving an average of a 55% hit ratio increase over LRU and PROBABIL-ITY GRAPH, a 36% hit ratio gain over AMP at reasonable cost. We further show that MITHRIL can supplement any cache replacement algorithm and be readily integrated into existing systems. Furthermore, we demonstrate the improvement comes from MITHRIL being able to capture mid-frequency blocks.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.