On-the-fly pipeline parallelism

Abstract

Pipeline parallelism organizes a parallel program as a linear sequence of s stages. Each stage processes elements of a data stream, passing each processed data element to the next stage, and then taking on a new element before the subsequent stages have necessarily completed their processing. Pipeline parallelism is used especially in streaming applications that perform video, audio, and digital signal processing. Three out of 13 benchmarks in PARSEC, a popular software benchmark suite designed for shared-memory multiprocessors, can be expressed as pipeline parallelism.Whereas most concurrency platforms that support pipeline parallelism use a "construct-and-run" approach, this paper investigates "on-the-fly" pipeline parallelism, where the structure of the pipeline emerges as the program executes rather than being specified a priori. On-the-fly pipeline parallelism allows the number of stages to vary from iteration to iteration and dependencies to be data dependent. We propose simple linguistics for specifying on-the-fly pipeline parallelism and describe a provably efficient scheduling algorithm, the PIPER algorithm, which integrates pipeline parallelism into a work-stealing scheduler, allowing pipeline and fork-join parallelism to be arbitrarily nested. The PIPER algorithm automatically throttles the parallelism, precluding "runaway" pipelines. Given a pipeline computation with T 1 work and T ∞ span (critical-path length), PIPER executes the computation on P processors in T P ≤ T 1 /P + O(T ∞ + lg P) expected time. PIPER also limits stack space, ensuring that it does not grow unboundedly with running time.We have incorporated on-the-fly pipeline parallelism into a Cilkbased work-stealing runtime system. Our prototype Cilk-P implementation exploits optimizations such as lazy enabling and dependency folding. We have ported the three PARSEC benchmarks that exhibit pipeline parallelism to run on Cilk-P. One of these, x264, cannot readily be executed by systems that support only construct-and-run pipeline parallelism. Benchmark results indicate that Cilk-P has low serial overhead and good scalability. On x264, for example, Cilk-P exhibits a speedup of 13.87 over its respective serial counterpart when running on 16 processors.

Using Template Matching to Infer Parallel Design Patterns

“…The pipeline detected by our approach can also be enhanced using the same techniques as Huang et al Rul et al [2007] uses the approach of Raman et al on the level of functions. Lee et al [2013] transform user-annotated code into pipelines using Cilk. Given that our approach can identify the code locations of pipeline stages automatically, it could help users of the approach of Lee et al to quickly find the places to annotate.…”

“…Following the observation that most parallel patterns are based on assumptions about data dependences, we replace UML diagrams with data-dependence graphs to characterize both the pattern and the software. Our approach, which is implemented as an extension of the data dependence profiler DiscoPoP [Li et al 2013;Li et al 2015], automatically infers potential parallel design patterns from the dependence graph of the program and specifies the division of code blocks according to the pattern structure. Based on this information, the programmer can then easily parallelize the code by moving suggested code blocks into appropriate structures of the pattern.…”

“…To obtain the dependence graph of a sequential program, we use an enhanced version of the dependence profiler DiscoPoP [Li et al 2013;Li et al 2015]. The version used in this study is based on LLVM [Lattner and Adve 2004] and follows a hybrid (i.e., combined static and dynamic) dependence analysis approach.…”

Race Detection in Two Dimensions

“…This restriction easily captures useful forms of pipeline parallelism [15], and so our race detector can be directly applied to analyze such pipelined programs. Our work can be seen as a generalization of existing race detectors for SP graphs to richer classes of graphs and language constructs.…”

“…However, their model is more relaxed than ours as it allows non-linear pipelines, therefore leaving open the question for efficient race detection in their case. Linear pipelines are the focus of the work of Lee et al [15] which extends Cilk with support for this setting. Interestingly, their language constructs are easily expressible in our restricted fork-join, but not the other way around, even though both models can express exactly the same task graphs, i.e., the ones having a two-dimensional lattice structure.…”