Twig: Profile-Guided BTB Prefetching for Data Center Applications

Abstract

Modern data center applications have deep software stacks, with instruction footprints that are orders of magnitude larger than typical instruction cache (I-cache) sizes. To efficiently prefetch instructions into the I-cache despite large application footprints, modern server-class processors implement a decoupled frontend with Fetch Directed Instruction Prefetching (FDIP). In this work, we first characterize the limitations of a decoupled frontend processor with FDIP and find that FDIP suffers from significant Branch Target Buffer (BTB) misses. We also find that existing techniques (e.g., stream prefetchers and predecoders) are unable to mitigate these misses, as they rely on an incomplete understanding of a program's branching behavior.To address the shortcomings of existing BTB prefetching techniques, we propose Twig, a novel profile-guided BTB prefetching mechanism. Twig analyzes a production binary's execution profile to identify critical BTB misses and inject BTB prefetch instructions into code. Additionally, Twig coalesces multiple non-contiguous BTB prefetches to improve the BTB's locality. Twig exposes these techniques via new BTB prefetch instructions. Since Twig prefetches BTB entries without modifying the underlying BTB organization, it is easy to adopt in modern processors. We study Twig's behavior across nine widely-used data center applications, and demonstrate that it achieves an average 20.86% (up to 145%) performance speedup

Thermometer

“…Shotgun [83] faces a slight slowdown as it wastes critical BTB capacity to store unused prefetch metadata. We corroborate the findings [23,75,132] of recent work and identify three reasons behind the performance degradation induced by these prior BTB prefetching policies [73,83]. First, like any temporal prefetcher [139][140][141],…”

“…As also observed by recent work [75,132], we find in Fig. 4 that Confluence [73] [73,83] provide merely 1.4% mean speedup, while a perfect BTB offers 63.2% mean speedup.…”

“…These results indicate that with a perfect BTB, FDIP can provide more benefits than with a perfect I-cache or a perfect branch predictor. Hence, optimizing BTB performance is critical to eliminate frontend stalls in the most efficient manner (as also reported by prior work [75,83,84,132]).…”

“…Prior work from Google and Facebook shows that their widely-deployed data center applications lose more than 15% of all pipeline slots due to frontend stalls [25,27,67,133]. As these applications are proprietary, we use the applications used by prior work [75,77,78,86,100,138,150], where frontend stalls are similarly frequent (more than 15%) due to large instruction footprints. These applications include cassandra [2], kafka [3], and tomcat [4] from the Java DaCapo benchmark suite [31], drupal [142], wordpress [144], and mediawiki [143] from Facebook's OSS -performance benchmark suite [16], finagle-chirper and finagle-http [12] from the Java Renaissance benchmark suite [114], clang [6] while building LLVM [85], PostgreSQL [10] while serving pgbench [9] queries, Python [14] while running the pyperformance [11] benchmark suite, MySQL [146] while serving TPC-C queries [35], and verilator [13] while emulating the Rocket Chip [7].…”

“…However, FDIP performs well only as long as the Branch Target Buffer (BTB) supplies correct targets for all taken branches [23,44,75,83,84,132]. Prior works have found that FDIP's performance is significantly limited by BTB misses that stall FDIP's prefetching [23,75,83,84] or cause FDIP to prefetch incorrect instructions on the wrong path [44,132].…”