Caffeine

Abstract

Aggressive prefetching improves system performance by hiding and tolerating off-chip memory latency. However, on a multicore system, prefetchers of different cores contend for shared resources and aggressive prefetching can degrade the overall system performance. The role of a prefetcher aggressiveness engine is to select appropriate aggressiveness levels for each prefetcher such that shared resource contention caused by prefetchers is reduced, thereby improving system performance. State-of-the-art prefetcher aggressiveness engines monitor metrics such as prefetch accuracy, bandwidth consumption, and last-level cache pollution. They use carefully tuned thresholds for these metrics, and when the thresholds are crossed, they trigger aggressiveness control measures. These engines have three major shortcomings: (1) thresholds are dependent on the system configuration (cache size, DRAM scheduling policy, and cache replacement policy) and have to be tuned appropriately, (2) there is no single threshold that works well across all the workloads, and (3) thresholds are oblivious to the phase change of applications. To overcome these shortcomings, we propose CAFFEINE, a model-based approach that analyzes the effectiveness of a prefetcher and uses a metric called net utility to control the aggressiveness. Our metric provides net processor cycles saved because of prefetching by approximating the cycles saved across the memory subsystem, from last-level cache to DRAM. We evaluate CAFFEINE across a wide range of workloads and compare it with the state-of-theart prefetcher aggressiveness engine. Experimental results demonstrate that, on average (geomean), CAFFEINE achieves 9.5% (as much as 38.29%) and 11% (as much as 20.7%) better performance than the best-performing aggressiveness engine for four-core and eight-core systems, respectively.

Band-Pass Prefetching

“…-We identify key issues with the state-of-the-art prefetcher aggressiveness control mechanisms: HPAC [Ebrahimi et al 2009] and CAFFEINE [Panda and Balachandran 2015] (Section 2). -We make two fundamental observations: (i) prefetch-accuracy strongly correlates with the ratio of number of L2 prefetch requests generated to an application's total requests (demands and prefetches from L1 and L2 caches) and (ii) prefetcher-caused interference (delay on demand misses) strongly correlates with the ratio of total prefetch requests to the total demands in the system (Section 3).…”

“…Prior works on prefetching focused both on exploiting simple sequential access patterns (e.g., Smith [1978], Jouppi [1990], and Palacharla and Kessler [1994]) in applications as well as on complex, nonsequential access patterns [Joseph and Grunwald 1997;Lai et al 2001;Nesbit and Smith 2004;Pugsley et al 2014;Shevgoor et al 2015;Michaud 2016]. While several works focused on maximizing the benefit of prefetching, some other works studied the interference caused by prefetching both in single-core [Zhuang and Lee 2003;Srinath et al 2007;Lin 2006, 2009;Wu et al 2011;Seshadri et al 2015] and multicore [Ebrahimi et al 2009[Ebrahimi et al , 2011Panda and Balachandran 2015;Jimenez et al 2015;Panda 2016;Ishii et al 2012] contexts. In this section, we discuss works that are close to our work.…”

“…However, it may cause severe interference at the shared resources (last level cache and memory bandwidth) of a multi-core system [Ebrahimi et al 2009[Ebrahimi et al , 2011Wu et al 2011;Seshadri et al 2015;Panda and Balachandran 2015;Jimenez et al 2015;Panda 2016;Lee et al 2008;Liu and Solihin 2011;Bitirgen et al 2008]. To manage prefetching in multi-core systems, prior studies [Srinath et al 2007;Ebrahimi et al 2009Ebrahimi et al , 2011Panda and Balachandran 2015;Panda 2016] have been proposed to dynamically control (also known as throttling) the prefetcher aggressiveness by adjusting the prefetcher-configuration at runtime. These mechanisms make dynamic throttling decisions by computing several parameters, such as prefetch-accuracy, lateness, prefetcher-caused interference at the last level cache and Dynamic Random-Access Memory (DRAM) in the form of pollution, row-buffer, bus, and bank conflicts.…”

“…Problem: Prior works such as Hierarchical Prefetcher Aggressiveness Control (HPAC) [Ebrahimi et al 2009] and CAFFEINE [Panda and Balachandran 2015] do not completely alleviate the problem of prefetcher-caused interference in multi-core systems. With HPAC, we observe that the use of multiple metrics (driven by their threshold values) does not capture the actual interference in the system, and in most cases leads to incorrect throttling decisions.…”

“…This section provides a background on our baseline system and the definitions that we use throughout the article. We then briefly describe HPAC [Ebrahimi et al 2009] and CAFFEINE [Panda and Balachandran 2015], two state-of-the-art prefetcher aggressiveness control mechanisms.…”