mPart: miss-ratio curve guided partitioning in key-value stores

Abstract

Web applications employ key-value stores to cache the data that is most commonly accessed. The cache improves an web application's performance by serving its requests from memory, avoiding fetching them from the backend database. Since the memory space is limited, maximizing the memory utilization is a key to delivering the best performance possible. This has lead to the use of multi-tenant systems, allowing applications to share cache space. In addition, application data access patterns change over time, so the system should be adaptive in its memory allocation. In this thesis, we address both multi-tenancy (where a single cache is used for multiple applications) and dynamic workloads (changing access patterns) using a model that relates the cache size to the application miss ratio, known as a miss ratio curve. Intuitively, the larger the cache, the less likely the system will need to fetch the data from the database. Our efficient, online construction of the miss ratio curve allows us to determine a near optimal memory allocation given the available system memory, while adapting to changing data access patterns. We show that our model outperforms an existing state-of-the-art sharing model, Memshare, in terms of cache hit ratio and does so at a lower time cost. We show that average hit ratio is consistently 1 percentage point greater and 99.9th percentile latency is reduced by as much as 2.9% under standard web application workloads containing millions of requests. xv

A Relational Theory of Locality

“…The fill time is mainly a metric for explaining the footprint theory. AET has been used in managing and optimizing the memory and storage cache (Byrne et al 2018;Hu et al 2018;Xiang et al 2018). For static analysis, found it more convenient to use AET, because it required only the reuse interval, not the first-and last-access times.…”

Faster slab reassignment in memcached

“…Third, we create realistic, multi-tenant, time-varying workload based rates measured at Facebook [1]. Finally, we run an another multi-tenant workload used in [4] that contains a diverse set of application access patterns and working set sizes. Our experiments run on a 48-core 2.2GHz Intel(R) Xeon(R) CPU E5-2650 v5 and Table 1 shows a comparison of CPU usage of the reassignment thread under different sleep intervals.…”

“…In some environments, multiple applications share a single memcached instance. This is referred to as multi-tenancy [4,8]. It allows applications to share the same cache resources as their demand for cache space changes over time.…”

“…One way to address this is to adjust the percentage of slabs that are reassigned at every interval. A typical limit is 10-20% depending on the application's cache behavior [4]. This also requires increasing the number of reassignment intervals in order to reach the best allocation for the workload.…”

“…With multi-tenant workloads (single cache, multiple applications [4]) and changing object size distributions common in most datacenter environments, it is also necessary to move memory between classes as quickly as possible so that application performance is not disturbed. We improve the slab reassignment mechanism in memcached to enable fast slab movement between classes.…”