Compiler-based prefetching for recursive data structures

Abstract

Software-controlled data prefetching offers the potential for bridging the ever-increasing speed gap between the memory subsystem and today's high-performance processors. While prefetching has enjoyed considerable success in array-based numeric codes, its potential in pointer-based applications has remained largely unexplored. This paper investigates compiler-based prefetching for pointer-based applications---in particular, those containing recursive data structures. We identify the fundamental problem in prefetching pointer-based data structures and propose a guideline for devising successful prefetching schemes. Based on this guideline, we design three prefetching schemes, we automate the most widely applicable scheme ( greedy prefetching ) in an optimizing research compiler, and we evaluate the performance of all three schemes on a modern superscalar processor similar to the MIPS R10000. Our results demonstrate that compiler-inserted prefetching can significantly improve the execution speed of pointer-based codes---as much as 45% for the applications we study. In addition, the more sophisticated algorithms (which we currently perform by hand, but which might be implemented in future compilers) can improve performance by as much as twofold. Compared with the only other compiler-based pointer prefetching scheme in the literature, our algorithms offer substantially better performance by avoiding unnecessary overhead and hiding more latency.

Automatic Prefetching by Traversal Profiling in Object Persistence Architectures

“…Luk and Mowry [20] have looked at optimizing recursive data structure access by predicting which parts of the structure will be accessed in the future. One of their approaches, history pointers, is similar in philosophy to our traversal profiles.…”

A Compile/Run-time Environment for the Automatic Transformation of Linked List Data Structures

“…By issuing a prefetch instruction before other work is done in a loop, memory latency caused by a cache-miss on a pointer traversal can be hidden, provided that the work done in the loop body is substantial. Karlsson et al [19] describe techniques to prefetch irregular accessed linked structures which extends the work of Luk and Mowry [24], who mention data linearization as a technique to improve prefetching efficiency and locality. Yang and Lebeck [33] present a memory architecture which pro-actively dereferences pointers lower in the hierarchy to push data higher into the memory hierarchy instead of waiting for the higher memory-level to pull the data.…”

Evaluating support for global address space languages on the Cray X1

“…Several studies [30,17,6] have proposed global communication scheduling techniques that attempt to find an optimal arrangement for all non-blocking memory accesses. Other variants of this optimization such as message strip mining [27] and software prefetching [18] are also useful in reducing an application's stall times due to communication latencies.…”