Algorithm + strategy = parallelism

Abstract

The process of writing large parallel programs is complicated by the need to specify both the parallel behaviour of the program and the algorithm that is to be used to compute its result. This paper introduces evaluation strategies: lazy higher-order functions that control the parallel evaluation of non-strict functional languages. Using evaluation strategies, it is possible to achieve a clean separation between algorithmic and behavioural code. The result is enhanced clarity and shorter parallel programs. Evaluation strategies are a very general concept: this paper shows how they can be used to model a wide range of commonly used programming paradigms, including divide-and-conquer parallelism, pipeline parallelism, producer/consumer parallelism, and data-oriented parallelism. Because they are based on unrestricted higher-order functions, they can also capture irregular parallel structures. Evaluation strategies are not just of theoretical interest: they have evolved out of our experience in parallelising several large-scale parallel applications, where they have proved invaluable in helping to manage the complexities of parallel behaviour. Some of these applications are described in detail here. The largest application we have studied to date, Lolita, is a 40,000 line natural language engineering system. Initial results show that for these programs we can achieve acceptable parallel performance, for relatively little programming effort.

Parallel and Concurrent Programming in Haskell

“…-Trinder et al [14], which has a wide range of examples. However beware: this paper is based on the earlier version of Strategies, and some of the examples may no longer work due to the new GC behaviour on sparks; also some of the names of functions and types in the library have since changed.…”

Eden – Parallel Functional Programming with Haskell

“…Moreover, access to local and remote data is automatically managed by the runtime system. Evaluation strategies [57,41] abstract from low-level expression marking and allow to describe patterns for parallel behaviour on a higher level of abstraction.…”

Proof Tool Support for Explicit Strictness

“…The purpose of this function is to fully reduce its argument and return True afterwards. Such an 'eval' function is usually used to express evaluation strategies in the context of parallelism [4,17]. We use eval for expressing definedness conditions.…”