Sandbox Prefetching: Safe run-time evaluation of aggressive prefetchers

Pugsley, Seth H.; Chishti, Zeshan; Wilkerson, Chris; Chuang, Peng-fei; Scott, Robert L.; Jaleel, Aamer; Lu, Shih-Lien; Chow, Kingsum; Balasubramonian, Rajeev

doi:10.1109/hpca.2014.6835971

Cited by 89 publications

(55 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Based on a speculative control flow path from a branch predictor, B-Fetch computes the effective address of the load instructions along that path based on a history of past register transformations and uses a per-load filtering mechanism to reduce cache pollution. Sandbox Prefetching [Pugsley et al 2014] combined the ideas of global pattern confirmation and immediate prefetching action to achieve high performance. It tracked candidate prefetch patterns at runtime by adding the prefetch addresses to a Bloom filter rather than actually fetching the data into the cache.…”

Section: Related Workmentioning

confidence: 99%

Thread-Aware Adaptive Prefetcher on Multicore Systems

Liu

Huang

2016

ACM Trans. Archit. Code Optim.

View full text Add to dashboard Cite

Most processors employ hardware data prefetching techniques to hide memory access latencies. However, the prefetching requests from different threads on a multicore processor can cause severe interference with prefetching and/or demand requests of others. The data prefetching can lead to significant performance degradation due to shared resource contention on shared memory multicore systems. This article proposes a thread-aware data prefetching mechanism based on low-overhead runtime information to tune prefetching modes and aggressiveness, mitigating the resource contention in the memory system. Our solution has three new components: (1) a self-tuning prefetcher that uses runtime feedback to dynamically adjust data prefetching modes and arguments of each thread, (2) a filtering mechanism that informs the hardware about which prefetching request can cause shared data invalidation and should be discarded, and (3) a limiter thread acceleration mechanism to estimate and accelerate the critical thread which has the longest completion time in the parallel region of execution. On a set of multithreaded parallel benchmarks, our thread-aware data prefetching mechanism improves the overall performance of 64-core system by 13% over a multimode prefetch baseline system with two-level cache organization and conventional modified, exclusive, shared, and invalid-based directory coherence protocol. We compare our approach with the feedback directed prefetching technique and find that it provides 9% performance improvement on multicore systems, while saving the memory bandwidth consumption.

show abstract

Section: Related Workmentioning

confidence: 99%

Thread-Aware Adaptive Prefetcher on Multicore Systems

Liu

Huang

2016

ACM Trans. Archit. Code Optim.

View full text Add to dashboard Cite

show abstract

“…Several works in recent years have proposed novel hardware prefetching schemes for improving the utilization of shared resources (offchip bandwidth and LLC capacity) in multicores [2,3,4,7,17,22]. Liu and Solihin [12] have proposed analytical models for bandwidth partitioning to identify when prefetching can help in improving system performance.…”

Section: Related Workmentioning

confidence: 99%

“…Several works have proposed hardware solutions for resource friendly prefetching [3,17,21] and shared resource management [16,22]. In this work we propose a runtime system with the ability to dynamically combine any implementation of software/hardware prefetching to improve performance on existing commodity processors instead of introducing a new hardware method.…”

Section: Related Workmentioning

confidence: 99%

“…While hardware prefetchers can improve single-thread performance considerably, aggressive prefetchers waste shared resources, such as offchip bandwidth and last-level cache capacity, which can impact overall processor performance [7,8,9,17]. Aggressive hardware prefetching wastes shared resources due to the trade-off between prefetch accuracy and prefetch coverage − to prefetch more useful cache lines, the prefetcher also fetches more useless cache lines [17]. Figure 1 shows that hardware prefetchers on an Intel Sandybridge processor significantly increase offchip data volume (useless data prefetching) and offchip bandwidth across most of the 14 memory-intensive Spec CPU 2006 benchmarks [6].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

AREP: Adaptive Resource Efficient Prefetching for Maximizing Multicore Performance

Khan

Laurenzanoy

Marsy

et al. 2015

2015 International Conference on Parallel Architecture and Compilation (PACT)

View full text Add to dashboard Cite

Abstract-Modern processors widely use hardware prefetching to hide memory latency. While aggressive hardware prefetchers can improve performance significantly for some applications, they can limit the overall performance in highlyutilized multicore processors by saturating the offchip bandwidth and wasting last-level cache capacity. Co-executing applications can slowdown due to contention over these shared resources.This work introduces Adaptive Resource Efficient Prefetching (AREP) − a runtime framework that dynamically combines software prefetching and hardware prefetching to maximize throughput in highly utilized multicore processors. AREP achieves better performance by prefetching data in a resource efficient way − conserving offchip-bandwidth and last-level cache capacity with accurate prefetching and by applying cache-bypassing when possible. AREP dynamically explores a mix of hardware/software prefetching policies, then selects and applies the best performing policy. AREP is phase-aware and re-explores (at runtime) for the best prefetching policy at phase boundaries.A multitude of experiments with workload mixes and parallel applications on a modern high performance multicore show that AREP can increase throughput by up to 49% (8.1% on average). This is complemented by improved fairness, resulting in average quality of service above 94%.

show abstract

“…With so much of the available on-die resources invested in the cache hierarchy, an efficient, high performance design requires intelligent cache management techniques. While many cache management and speculation techniques such as alternate replacement policies [6,14,15,18,27], deadblock/hit prediction [17,20,28,33,36], and prefetching techniques [2,10,16,19,19,23,26,31,32] have been extensively explored, many of these are piecemeal, one-off solutions that often interact poorly when implemented together and typically only address one level of the memory-system hierarchy. There has been little work exploring the interactions between these policies across multiple levels of the memory hierarchy and examining the information needed across boundaries in the system from software to the core, to the last level cache.…”

Section: Introductionmentioning

confidence: 99%

Kill the Program Counter

et al. 2017

Self Cite

View full text Add to dashboard Cite

Data prefetching and cache replacement algorithms have been intensively studied in the design of high performance microprocessors. Typically, the data prefetcher operates in the private caches and does not interact with the replacement policy in the shared Last-Level Cache (LLC). Similarly, most replacement policies do not consider demand and prefetch requests as different types of requests. In particular, program counter (PC)-based replacement policies cannot learn from prefetch requests since the data prefetcher does not generate a PC value. PC-based policies can also be negatively affected by compiler optimizations. In this paper, we propose a holistic cache management technique called Kill-the-PC (KPC) that overcomes the weaknesses of traditional prefetching and replacement policy algorithms. KPC cache management has three novel contributions. First, a prefetcher which approximates the future use distance of prefetch requests based on its prediction confidence. Second, a simple replacement policy provides similar or better performance than current state-of-the-art PC-based prediction using global hysteresis. Third, KPC integrates prefetching and replacement policy into a whole system which is greater than the sum of its parts. Information from the prefetcher is used to improve the performance of the replacement policy and vice-versa. Finally, KPC removes the need to propagate the PC through entire on-chip cache hierarchy while providing a holistic cache management approach with better performance than state-of-the-art PC-, and non-PC-based schemes. Our evaluation shows that KPC provides 8% better performance than the best combination of existing prefetcher and replacement policy for multi-core workloads.

show abstract

Sandbox Prefetching: Safe run-time evaluation of aggressive prefetchers

Cited by 89 publications

References 16 publications

Thread-Aware Adaptive Prefetcher on Multicore Systems

Thread-Aware Adaptive Prefetcher on Multicore Systems

AREP: Adaptive Resource Efficient Prefetching for Maximizing Multicore Performance

Kill the Program Counter

Contact Info

Product

Resources

About