Hybrid access-specific software cache techniques for the cell BE architecture

Abstract

Ease of programming is one of the main impediments for the broad acceptance of multi-core systems with no hardware support\ud for transparent data transfer between local and global memories. Software cache is a robust approach to provide the user with a\ud transparent view of the memory architecture; but this software approach can suffer from poor performance. In this paper, we propose a hierarchical, hybrid software-cache architecture that classifies at compile time memory accesses in two classes, highlocality\ud and irregular. Our approach then steers the memory references toward one of two specific cache structures optimized for their respective access pattern. The specific cache structures are optimized to enable high-level compiler optimizations to\ud aggressively unroll loops, reorder cache references, and/or transform surrounding loops so as to practically eliminate the\ud software cache overhead in the innermost loop. Performance evaluation indicates that improvements due to the optimized software-cache structures combined with the proposed codeoptimizations translate into 3.5 to 8.4 speedup factors, compared to a traditional software cache approach. As a result, we\ud demonstrate that the Cell BE processor can be a competitive alternative to a modern server-class multi-core such as the IBM\ud Power5 processor for a set of parallel NAS applications.Peer ReviewedPostprint (published version

Coherence protocol for transparent management of scratchpad memories in shared memory manycore architectures

“…Instead, the compiler has to wrap the potentially incoherent accesses with a piece of code that does a software lookup to check if some SPM has a copy of the data, triggers a fine-grained DMA transfer to bring the data to the SPM of the local core, accesses it, and triggers a DMA transfer back if the data is modified. This solution adds huge overheads, as observed in previous works in the area of software caches [22]. This paper avoids these overheads by allowing all the cores to access all the SPMs and by proposing a hierarchy of directories and filters that efficiently tracks the contents of all SPMs and diverts the potentially incoherent accesses to any SPM of the chip.…”

“…For a computational loop the code is transformed into a two-nested loop that uses tiling to do the computation [19,20,22,41], as shown in Figure 3. Each iteration of the outermost loop executes three phases: (1) a control phase that maps chunks of the array sections to the SPMs, (2) a synchronization phase that waits for the completion of the DMA transfers, and (3) a work phase that performs the computation for the currently mapped chunks of data.…”

“…The runtime library that manages the SPMs is a modified version of a software cache for the Cell B.E. [22] that has been ported to the hybrid memory system and optimized. Double buffering is not used.…”

“…A promising solution to exploit these characteristics in shared memory manycores is to introduce the hybrid memory system and to give the compiler the responsibility of generating code to manage the SPMs, so that the added programming complexity is not exposed to the programmer. Even though compilers succeed in generating code for the SPMs when the computation is based on predictable memory access patterns [22], in the presence of unpredictable memory accesses they encounter important limitations. Due to the incoherence between the SPMs and the cache hierarchy, the compiler cannot generate code for the SPMs if it cannot ensure that there is no aliasing between two memory references that may target copies of the same data in the SPMs and in the cache hierarchy.…”

Hardware-software coherence protocol for the coexistence of caches and local memories

“…Compilers succeed in generating code for LMs when the computation is based on predictable memory access patterns [7] but, when non-predictable memory access patterns are found, compilers need to ensure correctness by applying complex analyses such as memory aliasing [8], [9], [10]. When compilers cannot ensure that there is no aliasing between two memory references that may target copies of the same data in the LM and in the cache hierarchy, they must conservatively avoid using the LM.…”