Bouquet of Instruction Pointers: Instruction Pointer Classifier-based Spatial Hardware Prefetching

Pakalapati, Samuel; Panda, Biswabandan

doi:10.1109/isca45697.2020.00021

Cited by 43 publications

(33 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In Skip-List, the number of levels is 9. The number of lookups is 2 25 for all workloads. Each lookup finds exactly one node match with a uniformly distributed random unique key.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Fast Key-Value Lookups with Node Tracker

Cavus

Shatnawi

Sendag

et al. 2021

ACM Trans. Archit. Code Optim.

View full text Add to dashboard Cite

Lookup operations for in-memory databases are heavily memory bound, because they often rely on pointer-chasing linked data structure traversals. They also have many branches that are hard-to-predict due to random key lookups. In this study, we show that although cache misses are the primary bottleneck for these applications, without a method for eliminating the branch mispredictions only a small fraction of the performance benefit is achieved through prefetching alone. We propose the Node Tracker (NT), a novel programmable prefetcher/pre-execution unit that is highly effective in exploiting inter key-lookup parallelism to improve single-thread performance. We extend NT with branch outcome streaming (BOS) to reduce branch mispredictions and show that this achieves an extra 3× speedup. Finally, we evaluate the NT as a pre-execution unit and demonstrate that we can further improve the performance in both single- and multi-threaded execution modes. Our results show that, on average, NT improves single-thread performance by 4.1× when used as a prefetcher; 11.9× as a prefetcher with BOS; 14.9× as a pre-execution unit and 18.8× as a pre-execution unit with BOS. Finally, with 24 cores of the latter version, we achieve a speedup of 203× and 11× over the single-core and 24-core baselines, respectively.

show abstract

“…In Skip-List, the number of levels is 9. The number of lookups is 2 25 for all workloads. Each lookup finds exactly one node match with a uniformly distributed random unique key.…”

Section: Methodsmentioning

confidence: 99%

“…Recent hardware prefetchers targeting regular/repetitive patterns: Recent proposals in data prefetching include AAMPM [22], BO [16], SPP [17], ISB [18], VLDP [23], Bingo [24], and most recently, IPCP [25]. AMPM, BOP, and IPCP are the first-, second-and third-place winners of the 2019 data prefetching competition [26], respectively.…”

Section: Related Workmentioning

confidence: 99%

Fast Key-Value Lookups with Node Tracker

Cavus

Shatnawi

Sendag

et al. 2021

ACM Trans. Archit. Code Optim.

View full text Add to dashboard Cite

show abstract

“…This implementation doesn't use program or core counters, only with OS physical address space, but only improves performance for the state of the art over 6.4%. Other approaches can be found in [37] [38] [39].…”

Section: Other Prefetching Enginesmentioning

confidence: 99%

DeepP: Deep Learning Multi-Program Prefetch Configuration for the IBM POWER 8

Lurbe¹,

Feliu

Petit³

et al. 2022

IEEE Trans. Comput.

View full text Add to dashboard Cite

Current multi-core processors implement sophisticated hardware prefetchers, that can be configured by application (PID), to improve the system performance. When running multiple applications, each application can present different prefetch requirements, hence different configurations can be used. Setting the optimal prefetch configuration for each application is a complex task since it does not only depend on the application characteristics but also on the interference at the shared memory resources (e.g. memory bandwidth). In his paper, we propose DeepP, a deep learning approach for the IBM POWER8 that identifies at run-time the best prefetch configuration for each application in a workload. To this end, the neural network predicts the performance of each application under the studied prefetch configurations by using a set of performance events. The prediction accuracy of the network is improved thanks to a dynamic training methodology that allows learning the impact of dynamic changes of the prefetch configuration on performance. At run-time, the devised network infers the best prefetch configuration for each application and adjusts it dynamically. Experimental results show that the proposed approach improves performance, on average, by 5,8%, 6,7%, and 15,8% compared to the default prefetch configuration across different 6-, 8-, and 10-application workloads, respectively.

show abstract

“…Cooksey et al [23] present a content-directed prefetching mechanism to fetch any data brought into the cache that appears to be a pointer, regardless of whether it is used by the program, though compiler input [6,26] is typically necessary to throttle over-fetching. The Bouquet of Prefetchers [68] uses multiple different styles of prefetcher, selected by a PC-based predictor. IMP [88] targets stride-indirect memory access patterns in the cache, as do Takayashiki et al [79] via observing gathers.…”

Section: E Microarchitectural Prefetchersmentioning

confidence: 99%

Vector Runahead

Naithani

Ainsworth

Jones

et al. 2021

2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA)

View full text Add to dashboard Cite

The memory wall places a significant limit on performance for many modern workloads. These applications feature complex chains of dependent, indirect memory accesses, which cannot be picked up by even the most advanced microarchitectural prefetchers. The result is that current out-of-order superscalar processors spend the majority of their time stalled. While it is possible to build special-purpose architectures to exploit the fundamental memory-level parallelism, a microarchitectural technique to automatically improve their performance in conventional processors has remained elusive.Runahead execution is a tempting proposition for hiding latency in program execution. However, to achieve high memorylevel parallelism, a standard runahead execution skips ahead of cache misses. In modern workloads, this means it only prefetches the first cache-missing load in each dependent chain. We argue that this is not a fundamental limitation. If runahead were instead to stall on cache misses to generate dependent chain loads, then it could regain performance if it could stall on many at once. With this insight, we present Vector Runahead, a technique that prefetches entire load chains and speculatively reorders scalar operations from multiple loop iterations into vector format to bring in many independent loads at once. Vectorization of the runahead instruction stream increases the effective fetch/decode bandwidth with reduced resource requirements, to achieve high degrees of memory-level parallelism at a much faster rate. Across a variety of memory-latency-bound indirect workloads, Vector Runahead achieves a 1.79× performance speedup on a large out-of-order superscalar system, significantly improving on stateof-the-art techniques.

show abstract

Bouquet of Instruction Pointers: Instruction Pointer Classifier-based Spatial Hardware Prefetching

Cited by 43 publications

References 40 publications

Fast Key-Value Lookups with Node Tracker

Fast Key-Value Lookups with Node Tracker

DeepP: Deep Learning Multi-Program Prefetch Configuration for the IBM POWER 8

Vector Runahead

Contact Info

Product

Resources

About