Parallelizing Sequential Programs with Statistical Accuracy Tests

Abstract

We present QuickStep, a novel system for parallelizing sequential programs. QuickStep deploys a set of parallelization transformations that together induce a search space of candidate parallel programs. Given a sequential program, representative inputs, and an accuracy requirement, QuickStep uses performance measurements, profiling information, and statistical accuracy tests on the outputs of candidate parallel programs to guide its search for a parallelization that maximizes performance while preserving acceptable accuracy. When the search completes, QuickStep produces an interactive report that summarizes the applied parallelization transformations, performance, and accuracy results for the automatically generated candidate parallel programs. In our envisioned usage scenarios, the developer examines this report to evaluate the acceptability of the final parallelization and to obtain insight into how the original sequential program responds to different parallelization strategies. It is also possible for the developer (or even a user of the program who has no software development expertise whatsoever) to simply use the best parallelization out of the box without examining the report or further investigating the parallelization.Results from our benchmark set of applications show that QuickStep can automatically generate accurate and efficient parallel programs -the automatically generated parallel versions of five of our six benchmark applications run between 5.0 and 7.7 times faster on 8 cores than the original sequential versions. Moreover, a comparison with the Intel icc compiler highlights how QuickStep can effectively parallelize applications with features (such as the use of modern object-oriented programming constructs or desirable parallelizations with infrequent but acceptable data races) that place them inherently beyond the reach of standard approaches.

Automatic error elimination by horizontal code transfer across multiple applications

“…The check therefore introduces no new and potentially unpredictable behaviors -it simply narrows the set of inputs that the application decides to process. This narrowing is conceptually similar to transformations that eliminate concurrency errors by narrowing the set of possible interleavings [27,43].…”

“…But it is also possible to approach program behavior from an accuracy perspective (which characterizes program behaviors not as correct/incorrect or acceptable/unacceptable, but simply as more or less accurate). The QuickStep parallelizing compiler takes this perspective -it automatically generates parallel loops with data races, then applies accuracy-enhancing transformations (such as synchronization insertion and privatization) to increase the accuracy of the program and obtain an acceptably (but not necessarily completely) accurate approximate computation [43,44]. As approximate computing enters the mainstream, we expect accuracyenhancing transformations to become increasingly visible and important.…”

A significance-driven programming framework for energy-constrained approximate computing

“…Compared to implicitly parallel and explicitly parallel approximate computing frameworks, such as Quickstep [5], variability-aware OpenMP [8] and variation tolerant Open-MP [7], our programming and runtime system provides better optimization capability via selective execution of approximate code and dynamic control of accuracy and efficiency. We achieve this by providing concurrency control (elasticity) and the ability to execute in a range of operating points that meet quality and energy constraints.…”

Verifying quantitative reliability for programs that execute on unreliable hardware

“…Dynamic techniques observe the accuracy impact of program transformations [1,2,29,41,42,44,59,60,62,68], or injected soft errors [20,37,38,64,70]. Researchers have developed static techniques that use probabilistic reasoning to characterize the accuracy impact of various phenomena [6,18,43,47,56,72].…”