Practical Bounds on Optimal Caching with Variable Object Sizes

Abstract

Many recent caching systems aim to improve miss ratios, but there is no good sense among practitioners of how much further miss ratios can be improved. In other words, should the systems community continue working on this problem?Currently, there is no principled answer to this question. In practice, object sizes often vary by several orders of magnitude, where computing the optimal miss ratio (OPT) is known to be NP-hard. The few known results on caching with variable object sizes provide very weak bounds and are impractical to compute on traces of realistic length.We propose a new method to compute upper and lower bounds on OPT. Our key insight is to represent caching as a min-cost flow problem, hence we call our method the flow-based offline optimal (FOO). We prove that, under simple independence assumptions, FOO's bounds become tight as the number of objects goes to infinity. Indeed, FOO's error over 10M requests of production CDN and storage traces is negligible: at most 0.3%. FOO thus reveals, for the first time, the limits of caching with variable object sizes.While FOO is very accurate, it is computationally impractical on traces with hundreds of millions of requests. We therefore extend FOO to obtain more efficient bounds on OPT, which we call practical flow-based offline optimal (PFOO). We evaluate PFOO on several full production traces and use it to compare OPT to prior online policies. This analysis shows that current caching systems are in fact still far from optimal, suffering 11-43% more cache misses than OPT, whereas the best prior offline bounds suggest that there is essentially no room for improvement.

Performance evaluation for new web caching strategies combining LRU with score based object selection

“…We assume a lognormal distribution for object sizes with a mean of 622 kByte s k = e µ + σ Zk kByte with µ = 3.5, σ = 2.5, (6) where Z k is a standard normal distributed random variable. The size distribution is adapted to measurement statistics of cacheable CDN data chunks [10] and very similar results for web files reported in [74] [112][116][125] [130], which vary over a broad range from kByte to MByte. The popularity pk and the size sk of each object are assumed to be independent.…”

“…However, Belady's bound only applies for unit object size and for the hit count. Extensions for web caches with objects of different size and regarding caching values have not been addressed before 2018 [11][59], as to the authors' knowledge. Extended knapsack solutions to obtain maximum caching value are NP-hard, but standard knapsack heuristics are used to obtain upper and lower bounds around the maximum.…”

“…While the analysis becomes more complex for flexible and advanced caching methods, bounds for optimum caching efficiency have been derived from Belady's bound to general min-cost flow and knapsack solutions, which fully cover scorebased strategies [4][7] [11][19] [58][59] [92][93] [112] [116]. Time-to-live (TTL) caching emerged as another branch of strategies for web applications, where each object in a cache is valid only for a limited time.…”

“…• recently developed min-cost flow [11] and knapsack bounds [59], which overcome unit data size restrictions of Belady's bound [7], and thus apply to usual web cache scenarios with varying data size, and with a value or utility being assigned as a cost/benefit measure per cacheable data item, Figure 1: Scope of analysis methods for caching strategies (LFU: Least Frequently Used; GPU: Graphics Processing Unit)…”

“…They are extended at first to varying request rates over time, and finally to arbitrary request pattern in 2-dimensional (2D-)knapsack solutions [19] [59]. We compare the 2D-knapsack bound to similar results by Berger et al [11] via minimum cost flow optimization. The bounds even include score-and utility-based caching goals and methods [34][93].…”