Patterns and statistical analysis for understanding reduced resource computing

Abstract

We present several general, broadly applicable mechanisms that enable computations to execute with reduced resources, typically at the cost of some loss in the accuracy of the result they produce. We identify several general computational patterns that interact well with these resource reduction mechanisms, present a concrete manifestation of these patterns in the form of simple model programs, perform simulationbased explorations of the quantitative consequences of applying these mechanisms to our model programs, and relate the model computations (and their interaction with the resource reduction mechanisms) to more complex benchmark applications drawn from a variety of fields.

MOARD: Modeling Application Resilience to Transient Faults on Data Objects

“…For example, the multigrid solver can tolerate certain errors in hundreds of variables [14]. The essence of algorithm-level error masking is typically due to algorithm specific definition on execution fidelity and specific program constructs that mitigate error magnitude during application execution [16]. Limited analysis at individual operations or error propagation is not sufficient to build up a big picture to capture the algorithm-level fault tolerance.…”

Noise-Tolerant Explicit Stencil Computations for Nonuniform Process Execution Rates

“…These and similar techniques have been explored by others for different types of numerical methods. A very general survey of these ideas applied to scientific and other disciplines can be found in Rinard et al [2010] and Renganarayana et al [2012], and the citations therein. Such techniques remain an active area of research: see, for example, Donzis and Aditya [2014], and the citations therein.…”

“…In this context, nonuniformities in process runtimes have not been of interest to application developers. Research has instead been focused on maximizing data locality of a stencil to exploit cache and memory hierarchy size and structure as well as off chip bandwidth constraints in the case of single-process optimizations [Rosser 1998;Datta 2009;Wonnacott 2000;Douglas et al 2000;Strout et al 2001;Kamil et al 2006;Rivera and Tseng 2000;Mccalpin and Wonnacott 1999;Frigo and Strumpen 2005]. In the case of multiprocess optimizations, the interest lay in reducing communication costs [Rosser 1998;Chronopoulos and Gear 1989;Wonnacott 2000;Ding and He 2001;Demmel et al 2008;Ballard et al 2012;Georganas et al 2012].…”

“…Research has instead been focused on maximizing data locality of a stencil to exploit cache and memory hierarchy size and structure as well as off chip bandwidth constraints in the case of single-process optimizations [Rosser 1998;Datta 2009;Wonnacott 2000;Douglas et al 2000;Strout et al 2001;Kamil et al 2006;Rivera and Tseng 2000;Mccalpin and Wonnacott 1999;Frigo and Strumpen 2005]. In the case of multiprocess optimizations, the interest lay in reducing communication costs [Rosser 1998;Chronopoulos and Gear 1989;Wonnacott 2000;Ding and He 2001;Demmel et al 2008;Ballard et al 2012;Georganas et al 2012]. A great deal of research applicable to synchronization reduction has also been carried out in the development of formal techniques for compiler preprocessing of sequential codes for automatic parallelization, as well as in the development of preprocessing techniques for the optimization of existing parallel codes [Rosser 1998;Beletska et al 2010Beletska et al , 2011Wonnacott 2000].…”

“…In the case of multiprocess optimizations, the interest lay in reducing communication costs [Rosser 1998;Chronopoulos and Gear 1989;Wonnacott 2000;Ding and He 2001;Demmel et al 2008;Ballard et al 2012;Georganas et al 2012]. A great deal of research applicable to synchronization reduction has also been carried out in the development of formal techniques for compiler preprocessing of sequential codes for automatic parallelization, as well as in the development of preprocessing techniques for the optimization of existing parallel codes [Rosser 1998;Beletska et al 2010Beletska et al , 2011Wonnacott 2000]. Whatever the motivation or vocabulary, all of these approaches have in one way or another aimed to modify stencil computations in order to reduce synchronization.…”