I-SPY: Context-Driven Conditional Instruction Prefetching with Coalescing

Khan, Tanvir Ahmed; Sriraman, Akshitha; Devietti, Joseph; Pokam, Gilles; Litz, Heiner; Kasikci, Baris

doi:10.1109/micro50266.2020.00024

Cited by 23 publications

(11 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The analysis and slice extraction step takes on the order of 100 seconds. Previous works including Aut-oFDO [22], BOLT [88], AsmDB [7], I-SPY [59], Ripple [60], and Twig [58] have shown that profile-guided optimization techniques are practical and that they are deployed in data centers today.…”

Section: Methodsmentioning

confidence: 99%

CRISP: critical slice prefetching

Litz

Ayers

Ranganathan

2022

Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems

Self Cite

View full text Add to dashboard Cite

The high access latency of DRAM continues to be a performance challenge for contemporary microprocessor systems. Prefetching is a well-established technique to address this problem, however, existing implemented designs fail to provide any performance benefits in the presence of irregular memory access patterns. The hardware complexity of prior techniques that can predict irregular memory accesses such as runahead execution has proven untenable for implementation in real hardware. We propose a lightweight mechanism to hide the high latency of irregular memory access patterns by leveraging criticality-based scheduling. In particular, our technique executes delinquent loads and their load slices as early as possible, hiding a significant fraction of their latency. Furthermore, we observe that the latency induced by branch mispredictions and other high latency instructions can be hidden with a similar approach. Our proposal only requires minimal hardware modifications by performing memory access classification, load and branch slice extraction, as well as priority analysis exclusively in software. As a result, our technique is feasible to implement, introducing only a simple new instruction prefix while requiring minimal modifications of the instruction scheduler. Our technique increases the IPC of memory-latency-bound applications by up to 38% and by 8.4% on average. CCS CONCEPTS• Computer systems organization → Superscalar architectures.

show abstract

Section: Methodsmentioning

confidence: 99%

CRISP: critical slice prefetching

Litz

Ayers

Ranganathan

2022

Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…Prior work from Google and Facebook shows that their widely-deployed data center applications lose more than 15% of all pipeline slots due to frontend stalls [25,27,67,133]. As these applications are proprietary, we use the applications used by prior work [75,77,78,86,100,138,150], where frontend stalls are similarly frequent (more than 15%) due to large instruction footprints. These applications include cassandra [2], kafka [3], and tomcat [4] from the Java DaCapo benchmark suite [31], drupal [142], wordpress [144], and mediawiki [143] from Facebook's OSS -performance benchmark suite [16], finagle-chirper and finagle-http [12] from the Java Renaissance benchmark suite [114], clang [6] while building LLVM [85], PostgreSQL [10] while serving pgbench [9] queries, Python [14] while running the pyperformance [11] benchmark suite, MySQL [146] while serving TPC-C queries [35], and verilator [13] while emulating the Rocket Chip [7].…”

Section: Experimental Methodologymentioning

confidence: 99%

“…Moreover, they lose all the information about a branch every time the corresponding BTB entry is evicted. Since large working set sizes (both instruction and branch footprint) are the key characteristics of data center applications [27,67,75,77,78,132], it is necessary to retain branch reuse behavior even when the corresponding entry is not present in the BTB.…”

Section: Why Do Prior Replacement Policies Fall Short?mentioning

confidence: 99%

“…These techniques usually have one of two limitations: (1) the design is more complex [72,73,82] than real hardware prefetching implementations [119,122] or (2) the on-chip storage cost is too high [43,44]. Hybrid techniques [27,77] combine the effectiveness of software and hardware mechanisms but further increase the code footprint of data center applications. FDIP [118,119] offers a good trade-off among existing I-cache prefetchers as it provides performance comparable to state-of-theart prefetchers [22,46,48,51,99,101,120,127] without incurring a high storage cost [57,58].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Thermometer

Song

Khan

Shahri

et al. 2022

Proceedings of the 49th Annual International Symposium on Computer Architecture

Self Cite

View full text Add to dashboard Cite

Modern processors employ a decoupled frontend with Fetch Directed Instruction Prefetching (FDIP) to avoid frontend stalls in data center applications. However, the large branch footprint of data center applications precipitates frequent Branch Target Buffer (BTB) misses that prohibit FDIP from eliminating more than 40% of all frontend stalls. We find that the state-of-the-art BTB optimization techniques (e.g., BTB prefetching and replacement mechanisms) cannot eliminate these misses due to their inadequate understanding of branch reuse behavior in data center applications.In this paper, we first perform a comprehensive characterization of the branch behavior of data center applications, and determine that identifying optimal BTB replacement decisions requires considering both transient and holistic (i.e., across the entire execution) branch behavior. We then present Thermometer, a novel BTB replacement technique that realizes the holistic branch behavior via a profile-guided analysis. Based on the collected profile, Thermometer generates useful BTB replacement hints that the underlying hardware can leverage. We evaluate Thermometer using 13 widelyused data center applications and demonstrate that it provides an average speedup of 8.7% (0.4%-64.9%) while outperforming the state-of-the-art BTB replacement techniques by 5.6× (on average, the best performing prior work achieves 1.5% speedup). We also demonstrate that Thermometer achieves a performance speedup that is, on average, 83.6% of the speedup achieved by the optimal BTB replacement policy.

show abstract

“…Litz et al [26] present critical slice prefetching (CRISP) to prefetch hard-to-predict loads. Jamilan et al [19] and I-SPY [22] rely on Intel's LBR and dynamic execution information to optimize data and instruction prefetching, respectively. Ay-ers et al [4] study SPEC and Google workloads and present an automated way of classifying memory access patterns for software-based prefetching.…”

Section: Related Workmentioning

confidence: 99%

Workload Behavior Driven Memory Subsystem Design for Hyperscale

Mahar¹,

Wang²,

Shu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Hyperscalars run services across a large fleet of servers, serving billions of users worldwide. However, these services exhibit different behaviors compared to commonly available benchmark suites, leading to server architectures that are suboptimal for cloud workloads. As datacenters emerge as the primary server processor market, optimizing server processors for cloud workloads by better understanding their behavior is an area of interest. To address this, we present MemProf, a memory profiler that profiles the three major reasons for stalls in cloud workloads: code-fetch, memory bandwidth, and memory latency. We use MemProf to understand the behavior of cloud workloads at Meta and propose and evaluate micro-architectural and memory system design improvements that help cloud workloads' performance.MemProf's code analysis shows that cloud workloads at Meta execute the same code across CPU cores. Using this, we propose shared micro-architectural structures-a shared L2 I-TLB and a shared L2 cache. Next, to help with memory bandwidth stalls, using workloads' memory bandwidth distribution, we find that only a few pages contribute to most of the system bandwidth. We use this finding to evaluate a new high-bandwidth, small-capacity memory tier and show that it performs 1.46× better than the current baseline configuration. Finally, we look into ways to improve memory latency for cloud workloads. Profiling using MemProf reveals that L2 hardware prefetchers, which are commonly used to reduce memory latency, have very low coverage and consume a significant amount of memory bandwidth. To help improve future hardware prefetcher performance, we built an efficient memory tracing tool to collect and validate production memory access traces. Our memory tracing tool adds significantly less overhead than DynamoRIO, enabling tracing production workloads.

show abstract

I-SPY: Context-Driven Conditional Instruction Prefetching with Coalescing

Cited by 23 publications

References 48 publications

CRISP: critical slice prefetching

CRISP: critical slice prefetching

Thermometer

Workload Behavior Driven Memory Subsystem Design for Hyperscale

Contact Info

Product

Resources

About