Abstractions for portable, scalable parallel programming

Abstract

In parallel programming, the need to manage communication costs, load imbalance, and irregularities in the computation puts substantial demands on the programmer. Key properties of the architecture, such as the number of processors and the costs of communication, must be exploited to achieve good performance. Coding these properties directly into a program compromises the portability and exibility of the code because signicant changes are usually needed to port or enhance the program. We describe a parallel programming model that supports the concise, independent description of key aspects of a parallel program|such as data distribution, communication, and boundary conditions|without reference to machine idiosyncrasies. The independence of such components improves portability by allowing the components of a program to be tuned independently, and encourages reuse by supporting the composition of existing components. The architecture-sensitive aspects of a computation are isolated from the rest of the program, reducing the need to make extensive changes to port a program. This model is eective in exploiting both data parallelism and functional parallelism. This paper provides programming examples, compares this work to related languages, and presents performance results.

Experimental Evaluation of Automatic Array Alignment in Parallelized Matlab

“…A large suite of parallel array programming tools has been proposed, including array extensions to conventional languages [3], novel languages [1,2], and scalable linear algebra libraries [5].…”

ZPL: a machine independent programming language for parallel computers

“…For ZPL, the parallel analog to the imperative-procedural model is the Phase Abstractions programming model [4,34]. Phase Abstractions provide equivalent features to the imperative procedural model as well as data allocation and processor assignment information [4]. In a nutshell, the programming model defines a scalable unit of parallelism that encapsulates three aspects of parallel computations-code, data and communication-so that performance-critical characteristics of a parallel program can be adjusted for different target machines.…”

“…The remainder of the righthand side operates analogously, minimizing over the south and west arrays, and then combining these results into a single array of minimum values. 4 As mentioned earlier, the min function is a scalar C function that is promoted to operate on each element of an array expression.…”

“…The imperative-procedural model, typified by Fortran, C, Pascal, etc., has symbolic naming, data structures, procedures, parameters, recursion, control structures, etc. For ZPL, the parallel analog to the imperative-procedural model is the Phase Abstractions programming model [4,34]. Phase Abstractions provide equivalent features to the imperative procedural model as well as data allocation and processor assignment information [4].…”

Anahy: A Programming Environment for Cluster Computing

“…Overcoming the above difficulty is a challenge that involves both programming model [1] and scheduling. A common approach taken in the development of programming tools and runtime environments (e.g.…”