Predicting whole-program locality through reuse distance analysis

Abstract

Profiling can accurately analyze program behavior for select data inputs. We show that profiling can also predict program locality for inputs other than profiled ones. Here locality is defined by the distance of data reuse. Studying whole-program data reuse may reveal global patterns not apparent in short-distance reuses or local control flow. However, the analysis must meet two requirements to be useful. The first is efficiency. It needs to analyze all accesses to all data elements in full-size benchmarks and to measure distance of any length and in any required precision. The second is predication. Based on a few training runs, it needs to classify patterns as regular and irregular and, for regular ones, it should predict their (changing) behavior for other inputs. In this paper, we show that these goals are attainable through three techniques: approximate analysis of reuse distance (originally called LRU stack distance), pattern recognition, and distance-based sampling. When tested on 15 integer and floating-point programs from SPEC and other benchmark suites, our techniques predict with on average 94% accuracy for data inputs up to hundreds times larger than the training inputs. Based on these results, the paper discusses possible uses of this analysis.

Practical Loop Transformations for Tensor Contraction Expressions on Multi-level Memory Hierarchies

“…One key difference between the work in this area and our work is in the metrics used for transformations. Cache transformations are based on metrics like reuse distance [12] or stack distance [7]. In comparison, we target programmer controlled memory hierarchy levels, and applications where we primarily see capacity misses.…”

An Efficient Simulation Algorithm for Cache of Random Replacement Policy

“…However, for an important category of cache, namely those of random replacement policy, as the replacement policy randomly determines one among multiple candidates for eviction, the naïve simulation only constitutes a Monte Carlo simulation, therefore can only give one out of many possible results. In particular, in a single round, naïve simulation cannot give the hit probability of each cache reference, which is of particular interest in program analysis [3] [4]. If we maintain n copies of possible cache states and simulate access sequences on these states simultaneously, then the time and space requirement of the simulation will be equal to running n copies of naïve simulation in parallel, with no gain in efficiency.…”

“…2 Even if the replacement algorithm takes care to not evict valid data when there are free slots, as the cache is soon filled up with valid data, there will be no difference in practice. 3 Sometimes referred to as pseudo random replacement policy due to difficulty, if not impossibility, of obtaining true randomness.…”

The Potential of Computation Regrouping for Improving Locality

“…Some studied more structured programs and finer grained data-data reuses within and across loop nests [28], per-statement [29], and across program inputs [14]. These studies show that long-distance data reuses cause cache misses, but they do not show how well we can improve the locality of data access.…”

“…A trace may not represent the program behavior on other inputs, and a trace may be too large to be analyzed. For many programs, earlier work has shown that the temporal locality follows a predictable pattern and the (cache miss) behavior of all program inputs can be predicted by examining medium-size training runs [14,15,26,34,44]. In this paper, we use a medium-size input for each program.…”