Program Parallelization Using Synchronized Pipelining

Abstract

Abstract. While there are well-understood methods for detecting loops whose iterations are independent and parallelizing them, there are comparatively fewer proposals that support parallel execution of a sequence of loops or nested loops in the case where such loops have dependencies among them. This paper introduces a refined notion of independence, called eventual independence, that in its simplest form considers two loops, say loop 1 and loop 2 , and captures the idea that for every i there exists k such that the i + 1-th iteration of loop 2 is independent from the j-th iteration of loop 1 , for all j ≥ k. Eventual independence provides the foundation of a semantics-preserving program transformation, called synchronized pipelining, that makes execution of consecutive or nested loops parallel, relying on a minimal number of synchronization events to ensure semantics preservation. The practical benefits of synchronized pipelining are demonstrated through experimental results on common algorithms such as sorting and Fourier transforms.

Compiler and runtime support for the execution of scientific codes with unstructured datasets on heterogeneous parallel architectures

“…A survey of the most important compiler automation works for procedural languages has already been presented in Section 1.2.4. A relevant example for this Chapter is the work by Scandolo et al [SKH09], where the concept of Synchronized Pipelining is introduced to analyze the dependencies between different loops and determine which iterations of both loops can be run in parallel. Similarly to this Chapter, the approach is targeted at the parallelization of sequences of loops.…”