Program Sequentially, Carefully, and Benefit from Compiler Advances for Parallel Heterogeneous Computing

Abstract

International audienceThe current microarchitecture trend leads toward heterogeneity. This evolution is driven by the end of Moore's law and the frequency wall due to the power wall. Moreover, with the spreading of smartphone, some constraints from the mobile world drive the design of most new architectures. An immediate consequence is that an application has to be executable on various targets. Porting and maintaining multiple versions of the code base requires different skills and the efforts required in the process as well as the increased complexity in debugging and testing are time consuming, thus expensive. Some solutions based on compilers emerge. They are based either on directives added to C like in openhmpp or openacc or on automatic solution like pocc, Pluto, ppcg, or Par4All. However compilers cannot retarget in an efficient way any program written in a low-level language such as unconstrained C. Programmers should follow good practices when writing code so that compilers have more room to perform the transformations required for efficient execution on heterogeneous targets. This chapter explores the impact of different patterns used by programmers, and defines a set of good practices allowing a compiler to generate efficient code

Bridging the Performance Gap between Manual and Automatic Compilers with Intent-based Compilation

“…One main reason is that the automatically generated code, in the general case, can never be as efficiently optimized for execution on a particular architecture as hand-programmed code [3]. Indeed, the ability of a specialized human programmer to make complex code transformations judiciously by intuition and experience clearly defines the path that future parallel tools should take.…”

“…Traditional automatic compilers have continued to focus primarily on loops which, in general, happen to be the performance bottlenecks and the most compute intensive portions of sequential code. To correctly analyze such loops, the tools discourage the use of dynamic aliasing (where many pointers point to the same memory location) and complex and hybrid data types [3]. In addition, iterative versions of algorithms are preferred over recursive functions for easier analysis.…”

“…Automatic parallelization tools have continued to place restrictions on the sequential programmer. [2] and [3] have listed such suggestions, but they constrict the creativity of the sequential programmers. Moreover, these restrictions cannot be applied to the existing legacy sequential applications code base popularly used by the HPC and the Embedded System community.…”

“…The amount of speedup obtained using existing commercially available parallelization tools as compared to what can be potentially obtained with manual parallelization reflects this. Also, to obtain the highest possible performance for an application human intervention, sometimes rewriting code from scratch, will be required [3]. One lesson from years of research in parallel programming is that the complexity of parallel programming should be hidden from the programmer as far as possible [12].…”

“…This implementation is easily parallelized using automatic compilers and easily transformed into parallel code using code-to-code translators. The implementation of the algorithm using arrays is simple, encouraged [3] and makes the code readable, portable and maintainable.…”