Automatically Tuning Parallel and Parallelized Programs

Abstract

Abstract. In today's multicore era, parallelization of serial code is essential in order to exploit the architectures' performance potential. Parallelization, especially of legacy code, however, proves to be a challenge as manual efforts must either be directed towards algorithmic modifications or towards analysis of computationally intensive sections of code for the best possible parallel performance, both of which are difficult and time-consuming. Automatic parallelization uses sophisticated compiletime techniques in order to identify parallelism in serial programs, thus reducing the burden on the program developer. Similar sophistication is needed to improve the performance of hand-parallelized programs. A key difficulty is that optimizing compilers are generally unable to estimate the performance of an application or even a program section at compiletime, and so the task of performance improvement invariably rests with the developer. Automatic tuning uses static analysis and runtime performance metrics to determine the best possible compile-time approach for optimal application performance. This paper describes an offline tuning approach that uses a source-to-source parallelizing compiler, Cetus, and a tuning framework to tune parallel application performance. The implementation uses an existing, generic tuning algorithm called Combined Elimination to study the effect of serializing parallelizable loops based on measured whole program execution time, and provides a combination of parallel loops as an outcome that ensures to equal or improve performance of the original program. We evaluated our algorithm on a suite of hand-parallelized C benchmarks from the SPEC OMP2001 and NAS Parallel benchmarks and provide two sets of results. The first ignores hand-parallelized loops and only tunes application performance based on Cetus-parallelized loops. The second set of results considers the tuning of additional parallelism in hand-parallelized code. We show that our implementation always performs near-equal or better than serial code while tuning only Cetus-parallelized loops and equal to or better than hand-parallelized code while tuning additional parallelism.

Performance Analysis and Tuning of Automatically Parallelized OpenMP Applications

“…We compare the speedup of our tuning framework (Window Tuned) with the speedup of the serial code, hand parallel code, Cetus parallel code, and Cetus parallel code tuned by the automatic tuning framework presented in related work [11]. We will name the later CE, referring to the combined elimination algorithm implemented in its search space navigation.…”

“…We compare our framework to the automatic tuning framework presented in related work [11]. The tuning process in [11] starts with a fixed initial configuration point.…”

“…The tuning process in [11] starts with a fixed initial configuration point. The optimization configuration is provided to Cetus for version generation.…”

“…Some program sections, like small inner loops, are parallelizable but parallelizing them does not increase performance. Our framework currently tunes unprofitable loopparallelization [11] and function-inlining techniques by selectively applying these techniques to loop nests when they improve performance. Inlining is often applied as a pre-pass to optimization, making it difficult to tune.…”

Towards software performance engineering for multicore and manycore systems

“…Furthermore, it is difficult to determine actual memory consumption, itself an essential factor to garbage collection performance models [37]. The compiler construction research is currently aiming at techniques to automatically improve software for multicore environments, for example by parallelizing loop executions [38]. Once such techniques are commonly employed by compilers, MCSPE models will need to take thus gained speedup into account, e.g.…”