Harnessing parallelism in multicore clusters with the All-Pairs, Wavefront, and Makeflow abstractions

Abstract

Both distributed systems and multicore systems are difficult programming environments. Although the expert programmer may be able to carefully tune these systems to achieve high performance, the non-expert may struggle. We argue that high level abstractions are an effective way of making parallel computing accessible to the non-expert. An abstraction is a regularly structured framework into which a user may plug in simple sequential programs to create very large parallel programs. By virtue of a regular structure and declarative specification, abstractions may be materialized on distributed, multicore, and distributed multicore systems with robust performance across a wide range of problem sizes. In previous work, we presented the All-Pairs abstraction for computing on distributed systems of single CPUs. In this paper, we extend All-Pairs to multicore systems, and introduce the Wavefront and Makeflow abstractions, which represent a number of problems in economics and bioinformatics. We demonstrate good scaling of both abstractions up to 32 cores on one machine and hundreds of cores in a distributed system.

Adapting bioinformatics applications for heterogeneous systems: a case study

“…Given that the abstractions Weaver provides related well to those in PEMer (Figure 1), we found it relatively easy to refactor them into a Makeflow [12]. A sample of the code to do this is given in Table 1.…”

“…Frameworks such as our Makeflow engine [12] provide a straightforward route for software with clear data dependencies and simple serial execution. Because a single bioinformatics Makeflow can be thousands of lines long, we have shown that creating scripts and higher level programs is the key for successful and practical parallelization.…”

A Specification Language for Performance and Economical Analysis of Short Term Data Intensive Energy Management Services

“…Reuse of components allows developers to use knowledge of their properties to predict the new system properties [12]. (3) Workflow and data flow patterns have been widely studied [11,18]. More advanced data flow abstractions such as MapReduce, or Bulk Synchronous parallel (BSP) constitute the essence of parallel programming frameworks such as Hadoop and Hama.…”

“…To illustrate a more complex specification with Langliers we present the specification of an operational model for Matrix-Vector Multiplication in streaming based on the wavefront abstraction [18]. The wavefront abstraction is a paradigmatic regularly structured framework that allow developer to focus on simple sequential programs to create very large parallel programs.…”

“…It is illustrated by means of the MatrixVector Multiplication problem in streaming fashion, an example of the kind of advanced data flow abstractions required for real-time SG state estimations in the cloud. This problem can be adequately modeled by the wavefront pattern [18]. Section 3 analyses the use of different analysis possibilities of the Petri net (PN) underlying the model, and finally conclusions are given in Section 4.…”