Improving cache locality with blocked array layouts

Abstract

Minimizing cache misses is one of the most important factors to reduce average latency for memory accesses. Tiled codes modify the instruction stream to exploit cache locality for array accesses. In this paper, we further reduce cache misses, restructuring the memory layout of multidimensional arrays, that are accessed by tiled instruction code. In our method, array elements are stored in a blocked way, exactly as they are swept by the tiled instruction stream. We present a straightforward way to easily translate multidimensional indexing of arrays into their blocked memory layout using simple binary-mask operations. Indices for such array layouts are easily calculated based on the algebra of dilated integers, similarly to morton-order indexing. Actual experimental results, using matrix multiplication and LU-decomposition on various size arrays, illustrate that execution time is greatly improved when combining tiled code with tiled array layouts and binary maskbased index translation functions. Simulations using the Simplescalar tool, verify that enhanced performance is due to the considerable reduction of total cache misses.

Tile/line access cache memory based on a multi‐level Z‐order tiling data layout

“…(2) The approach cannot support simultaneous parallel small tile access. Although other studies() have used the Z‐Morton layout to exploit 2‐D data locality and reduce conflict misses, those approaches have increased the cycle time or the access latency due to the versatility of the Morton‐index translations. In contrast, Lim and Thottethodi proposed a hardware‐based bit‐permuting unit to translate the raster scan order address to a Z‐Morton address.…”