Design and evaluation of a compiler algorithm for prefetching

Mowry, Todd C.; Lam, Monica S.; Gupta, Anoop

doi:10.1145/143365.143488

Cited by 522 publications

(196 citation statements)

References 19 publications

Supporting

Mentioning

192

Contrasting

Unclassified

Order By: Relevance

“…Both data references and instruction references suffer from the increased memory latency. But, unlike the data references, which can often be scheduled by programmers or compilers to tolerate long memory latency or through software controlled data cache prefetch [10], instructions usually depend on the instruction cache to overcome long memory latency. 2) With parallel processing techniques that spread loops over multiple processors, loop counts become reduced, so temporal locality is reduced.…”

Section: Discussionmentioning

confidence: 99%

A performance study of instruction cache prefetching methods

Hsu

Smith

1998

IEEE Trans. Comput.

View full text Add to dashboard Cite

Abstract-Prefetching methods for instruction caches are studied via trace-driven simulation. The two primary methods are "fallthrough" prefetch (sometimes referred to as "one block lookahead") and "target" prefetch. Fall-through prefetches are for sequential line accesses, and a key parameter is the distance from the end of the current line where the prefetch for the next line is initiated. Target prefetches work also for nonsequential line accesses. A prediction table is used and a key aspect is the prediction algorithm implemented by the table. Fall-through prefetch and target prefetch each improve performance significantly. When combined in a hybrid algorithm, their performance improvement is nearly additive. An instruction cache using a combined target and fall-through method can provide the same performance as a two to four times larger cache that does not prefetch. A good prediction method must not only be accurate, but prefetches must be initiated early enough to allow time for the instructions to return from main memory. To quantify this, we define a "prefetch efficiency" measure that reflects the amount of memory fetch delay that may be successfully hidden by prefetching. The better prefetch methods (in terms of miss rate) also have very high efficiencies, hiding approximately 90 percent of the miss delay for prefetched lines. Another performance measure of interest is memory traffic. Without prefetching, large line sizes give better hit rates; with prefetching, small line sizes tend to give better overall hit rates. Because smaller line sizes tend to reduce memory traffic, the top-performing prefetch caches produce less memory traffic than the topperforming nonprefetch caches of the same size.

show abstract

Section: Discussionmentioning

confidence: 99%

A performance study of instruction cache prefetching methods

Hsu

Smith

1998

IEEE Trans. Comput.

View full text Add to dashboard Cite

show abstract

“…al [21] introduced a compiler algorithm to insert prefetch instructions into scientific applications that operate on dense matrices. Consider the code in Fig.…”

Section: Existing Software Prefetching Methodsmentioning

confidence: 99%

“…Software prefetching [21,19,7] relies on the existence of non-blocking prefetch instructions and is usually enabled by the compiler. In hardware prefetching (e.g.…”

Section: Related Workmentioning

confidence: 99%

Resource-Aware Compiler Prefetching for Fine-Grained Many-Cores

Caragea

Tzannes

Keceli

et al. 2011

Int J Parallel Prog

View full text Add to dashboard Cite

Abstract. Super-scalar, out-of-order processors that can have tens of read and write requests in the execution window place significant demands on Memory Level Parallelism (MLP). Multi-and many-cores with shared parallel caches further increase MLP demand. Current cache hierarchies however have been unable to keep up with this trend, with modern designs allowing only 4-16 concurrent cache misses. This disconnect is exacerbated by recent highly parallel architectures (e.g. GPUs) where power and area per-core budget favor numerous lighter cores with less resources, further reducing support for MLP on a per-core basis. Support for hardware and software prefetch increases MLP pressure since these techniques overlap multiple memory requests with existing computation. In this paper, we propose and evaluate a novel Resource-Aware Prefetching (RAP) compiler algorithm that is aware of the number of simultaneous prefetches supported, and optimized for the same. We implemented our algorithm in a GCC-derived compiler and evaluated its performance using an emerging fine-grained many-core architecture. Our results show that the RAP algorithm outperforms a well-known loop prefetching algorithm by up to 40.15% in run-time on average across benchmarks and the state-of-the art GCC implementation by up to 34.79%, depending upon hardware configuration. Moreover, we compare the RAP algorithm with a simple hardware prefetching mechanism, and show run-time improvements of up to 24.61%. To demonstrate the robustness of our approach, we conduct a designspace exploration (DSE) for the considered target architecture by varying (i) the amount of chip resources designated for per-core prefetch storage and (ii) off-chip bandwidth. We show that the RAP algorithm is robust in that it improves performance across all design points considered. We also identify the Pareto-optimal hardware-software configuration which delivers 53.66% run-time improvement on average while using only 5.47% more chip area than the bare-bones design.

show abstract

“…Prefetch requires loading a cache block before it is requested, either by hardware [7,20] or with compiler support [28]; predictive prefetch attempts to improve accuracy of prefetch for relatively varied memory access patterns [1]. In critical word first, the word containing the reference which caused the miss is fetched first, followed by the rest of the block [13].…”

Section: Alternativesmentioning

confidence: 99%

L1 Cache and TLB Enhancements to the RAMpage Memory Hierarchy

Machanick

Patel

2003

Advances in Computer Systems Architecture

View full text Add to dashboard Cite

The RAMpage hierarchy moves main memory up a level to replace the lowest-level cache by an equivalent-sized SRAM main memory, and uses the TLB to cache page translations in that main memory. Earlier RAMpage evaluation used a relatively small L1 cache and TLB. For the RAMpage hierarchy, the effect of a small TLB is clearer than with a conventional hierarchy, because it is more feasible to make a TLB which maps a high fraction of main memory pages. This paper illustrates how more aggressive components higher in the memory hierarchy make time spent waiting for DRAM more significant as a fraction of total execution time, and, hence, approaches to hide the latency of DRAM become more important. For an instruction issue rate of 1 GHz, the simulated standard hierarchy waited for DRAM 10% of the time; with the instruction issue rate increased to 8 GHz, the fraction of time spent waiting for DRAM increased to 40%, and was higher for a larger L1 cache. The RAMpage hierarchy with context switches on misses was able to hide almost all DRAM latency. Increasing the processor speed in a standard hierarchy by a factor of 8 and increasing L1 cache size by a factor of 16, with DRAM speed unchanged, resulted in a speedup of 6.12. Adding the RAMpage model and introducing context switches on misses, with similar processor speed and L1 improvements, resulted in a speedup of 10.7 over the slowest conventional hierarchy, or 1.74 for the best case of the conventional hierarchy with the fastest processor. A larger TLB was shown to increase the viable range of RAMpage SRAM page sizes.

show abstract

Design and evaluation of a compiler algorithm for prefetching

Cited by 522 publications

References 19 publications

A performance study of instruction cache prefetching methods

A performance study of instruction cache prefetching methods

Resource-Aware Compiler Prefetching for Fine-Grained Many-Cores

L1 Cache and TLB Enhancements to the RAMpage Memory Hierarchy

Contact Info

Product

Resources

About