Implementing Powerlists with Bulk Synchronous Parallel ML

Abstract

Tools and methods able to simplify the development process of parallel software, but also to assure a high level of performance and robustness, are necessary. Powerlists and their variants are data structures that can be successfully used in a simple, provably correct, functional description of parallel programs, which are divide-and-conquer in nature. The paper presents how programs defined based on powerlists could be implemented in the functional language OCaml plus calls to the parallel functional programming library Bulk Synchronous Parallel ML. BSML functions follow the BSP model requirements, and so its advantages are introduced in OCaml parallel code. In order to write powerlist programs in BSML we provide a datatype for powerlists and a set of skeletons (higher-order functions implemented in parallel) to manipulate them. Examples are given and concrete experiments for their executions are conducted. Keywords-Parallel recursive structures; Functional parallel programming; Bulk synchronous parallelism I. CONTEXT AND MOTIVATIONThe latest developments of the computation systems lead to an increase of the requirements in using parallel computation. Still, for many years, parallel computation has been considered difficult and error-prone. This imposes using tools and methodologies able to simplify the development process of parallel software, but also to assure a high level of performance and robustness.This calls for a strongly structured form of parallelism [1], [2], [3], which should not only be based on an abstraction or model that conceals much of the complexity of parallel computation, but also provide a systematic way of developing such parallelism from specifications for practically nontrivial examples. Since correctness is very important in this context, high-level algebraic theories are appropriate to be used as fundamentals. Among them at least two seem suitable for parallel programming: the theory of lists [4] and parallel recursive structures such as powerlists [5].Powerlists and their variants are data structures that can be successfully used in a simple, provably correct, functional description of parallel programs, which are divide and conquer in nature [5], [6]. For each data structure, theories based on algebras and structural induction principles have been specified, which make them well suited to formally define recursive, data-parallel algorithms. These theories can be considered together a base for a model of parallel computation with a very high level of abstraction [7].In order to be useful, a model of parallel computation must also address very carefully issues such as efficient implementation and costs evaluation. BSP model [2] is famous especially because it provides a very accurate cost analysis, and a rigorous development that could provide robustness. From these, the idea of using BSP development methodology in the process of implementing powerlists programs came natural.Powerlist programs are defined in a very high-level way. While their divide-and-conquer nature makes them suita...

PList-based Divide and Conquer Parallel Programming

“…They provide natural support for Divide & Conquer patterns based on the data domain decomposition. Prior existing work that tries to make use of the PowerList theory powerful abstraction are [18], [19] and [20]. In [18] authors introduce transformation rules over PowerLists functions in order to adapt the PowerLists programs for the massively data parallel model.…”

“…In [18] authors introduce transformation rules over PowerLists functions in order to adapt the PowerLists programs for the massively data parallel model. [19] shows a functional implementation of PowerList functions in BSML (Bulk Synchronous Parallel ML). Also PowerLists have also been used to capture parallelism and recursion succinctly for GPU computing [20].…”

Parallel Programming with OCaml: A Tutorial

“…BSML has been used for example to implement parallel data structures [34], algorithmic skeletons on distributed lists [35], and powerlists [36]. b) Multi-ML: In the tradition of BSML, Multi-ML aims to propose formal properties.…”

A BSPlib-style API for Bulk Synchronous Parallel ML

“…GADT are useful in the context of parallelism. We also used them for implementing recursive parallel data structures, namely Misra's powerlists [24], and associated operations in BSML while ensuring that no nesting of parallel vector occurs [18].…”