Scheduling reductions on realistic machines

Abstract

Many computations can be modeled with systems of affine recurrence equations (SAREs) over polyhedral domains. We study the problem of scheduling individual computations of an SARE in the presence of reductions i.e., operations specifying the accumulation of a set of values to produce a single value. Reductions involve a commutative and associative operator and therefore, per se, do not impose any specific order. However, on realistic machines, operators have bounded fan-in and therefore an order of accumulation (serialization) is needed. Arbitrary serializations may adversely affect the running time of a program. We develop an algorithm to determine efficient serializations of all reductions. We illustrate our methods with two significant examples.

Simplifying dependent reductions in the polyhedral model

“…A valid, dependence-satisfying schedule that satisfies Equation (8) may be non-sequential in that it permits multiple statement instances to execute in the same timestep. Therefore, it is possible for a schedule to demand an unbounded number of statement instances to execute at the same timestamp, which does not directly map to physical machines with finite resources [Gupta et al 2002;Redon and Feautrier 1994].…”

“…ST in the polyhedral IR for the example in Section 1 (Equation (4)), given the reuse vector [1, 0] ⊤ for all j, and therefore requires accumulation of a potentially unbounded number of values. Gupta et al [2002] demonstrate a scheduling approach that bounds the total accumulations per timestep to target physical machines.…”

Scheduling and Tiling Reductions on Realistic Machines

“…They assumed that accumulations can happen in one time step. This approach was extended by [Gupta et al 2002] to work on realistic machines. They invented a scheduling technique for machines with binary operators and exclusive writes.…”

“…Section 2 describes necessary background. In section 3, we describe [Gupta et al 2002] scheduling technique using examples. Section 4 exposes the flaw in their technique using a counter example which shows that "exclusive writes" condition is violated and proposes a solution that is guaranteed to give schedules for machines with "exclusive writes".…”

“…This makes scheduling reductions a difficult and interesting problem.There have been a number of research works in the context of scheduling reductions. We focus on the scheduling techniques presented in [Gupta et al 2002], identify a potential issue in their scheduling algorithm and provide a solution. In addition, we demonstrate how these scheduling techniques can be extended to "tile" reductions and briefly survey other studies that address the problem of scheduling reductions.…”