Value-Profile Guided Stride Prefetching for Irregular Code

Abstract

Memory operations in irregular code are difficult to prefetch, as the future address of a memory location is hard to anticipate by a compiler. However, recent studies as well as our experience indicate that many irregular programs contain loads with near-constant strides. This paper presents a novel compiler technique to profile and prefetch for those loads. The profile captures not only the dominant stride values for each profiled load, but also the differences between the successive strides of the load. The profile information helps the compiler to classify load instructions into strongly or weakly strided and singlestrided or phased multi-strided. The prefetching decisions guided by the load classifications are highly selective and beneficial. We obtain significant performance improvement for the CPU2000 integer programs running on Itanium machines. For example, we achieve a 1.55x speedup for "181.mcf", 1.15x for "254.gap", 1.08x for "197.parser" and smaller gains in other benchmarks. We also show that the performance gain is stable across profile data sets and that the profiling overhead is low. These benefits make the new technique suitable for a production compiler.

Efficient discovery of regular stride patterns in irregular programs and its use in compiler prefetching

“…However, we have yet to see them bringing positive overall performance gain for the SPECINT2000 benchmarks. For example, in [27] we established that the static prefetching method results in a negative overall performance improvement.…”

Light-PREM: Automated software refactoring for predictable execution on COTS embedded systems

“…The extension of such algorithm that exploits multiprocessing to mask memory latency is detailed in [15]. In [16,17] Wu et al propose a more general algorithm that is able to generate prefetch sequences by analyzing stride memory access patterns to handle both array-based and pointer-based structures. Efficient compiletime strategies to automatically insert prefetch statements, when pointer-based and recursive structures are in use, are studied in [18,19].…”

Prefetch injection based on hardware monitoring and object metadata

“…Prefetching linked data structures requires predicting the access patterns, a challenging problem. Luk et al [23] and Wu et al [34] use stride prefetching to exploit regular patterns of access in linked-list traversals. The literature for hardware stride prefetching is abundant; for example, see Sair et al [26].…”