A compiler optimization algorithm for shared-memory multiprocessors

Abstract

This paper presents a new compiler optimization algorithm that parallelizes applications for symmetric, shared-memory multiprocessors. The algorithm considers data locality, parallelism, and the granularity of parallelism. It uses dependence analysis and a simple cache model to drive its optimizations. It also optimizes across procedures by using interprocedural analysis and transformations. We validate the algorithm by hand-applying it to sequential versions of parallel, Fortran programs operating over dense matrices. The programs initially were hand-coded to target a variety of parallel machines using loop parallelism. We ignore the user's parallel loop directives, and use known and implemented dependence and interprocedural analysis to find parallelism. We then apply our new optimization algorithm to the resulting program. We compare the original parallel program to the hand-optimized program, and show that our algorithm improves three programs, matches four programs, and degrades one program in our test suite on a shared-memory, bus-based parallel machine with local caches. This experiment suggests existing dependence and interprocedural array analysis can automatically detect user parallelism, and demonstrates that user parallelized codes often benefit from our compiler optimizations, providing evidence that we need both parallel algorithms and compiler optimizations to effectively utilize parallel machines.

Author retrospective for optimizing for parallelism and data locality

“…Optimizing one loop in isolation was and is not sufficient for the best performance; data reorganization, loop fusion, and loop distribution add substantial benefits [7,10,11,16,21,22,25]. Our model directly computed the best permutations for locality and parallelism and we showed [7,16,20] how to directly derive good loop fusion and distribution choices, which the polyhedral model could not yet perform. Our approach also had the advantage that the resulting code was human readable and suitable for use in an interactive parallelization tool [17].…”

Parallel Low-Storage Runge—Kutta Solvers for ODE Systems with Limited Access Distance

“…Modern optimizing compilers try to achieve an efficient exploitation of the memory hierarchy by reordering the instructions of the source program [2,49]. A lot of the research in this area concentrates on computationally intensive loops [24,25,35], and loop tiling [4,44] is considered to be one of the most successful techniques.…”

Invited Paper: A Compile-time Cost Model for OpenMP

“…Ferrante et al [10] determines the innermost loop with overflowing caches using the number of distinct cache lines accessed inside a loop to guide transformations like loop interchange. McKinley's cache model [19] is based on equivalence classes of array references showing temporal and spatial locality. Ghosh et al [11] introduces cache miss equations based on a system of linear Diophantine equations from a reuse vector.…”