Efficient and correct execution of parallel programs that share memory

Abstract

In this paper we consider an optimization problem that arises in the execution of parallel programs on shared-memory multiple-instruction-stream, multiple-data-stream (MIMD) computers. A program on such machines consists of many sequential program segments, each executed by a single processor. These segments interact as they access shared variables. Access to memory is asynchronous, and memory accesses are not necessarily executed in the order they were issued. An execution is correct if it is sequentially consistent: It should seem as if all the instructions were executed sequentially, in an order obtained by interleaving the instruction streams of the processors. Sequential consistency can be enforced by delaying each access to shared memory until the previous access of the same processor has terminated. For performance reasons, however, we want to allow several accesses by the same processor to proceed concurrently.Our analysis finds a minimal set of delays that enforces sequential consistency. The analysis extends to interprocessor synchronization constraints and to code where blocks of operations have to execute atomically. We use a conflict graph similar to that used to schedule transactions in distributed databases. Our graph incorporates the order on operations given by the program text, enabling us to do without locks even when database conflict graphs would suggest that locks are necessary. Our work has implications for the design of multiprocessors; it offers new compiler optimization techniques for parallel languages that support shared variables.

Bounded Model Checking of Concurrent Data Types on Relaxed Memory Models: A Case Study

“…Specialized algorithms to insert memory fences automatically during compilation have been proposed [27,28]. However, these methods are based on a conservative program analysis, and they enforce sequential consistency on the instruction level rather than the operation level.…”

Memorax, a Precise and Sound Tool for Automatic Fence Insertion under TSO

“…In addition to the above update limitation, MEM-ORAX uses a partial order reduction technique based on the principle that an instruction reordering that does not participate in a conflict cycle, as defined in [13], can be simulated by an appropriate scheduling under SC. Thus instruction reorderings that do not participate in conflict cycles need not be analysed.…”

“…We perform an extensive analysis to capture fences that need to be added in order to avoid a given error trace. By identifying the conflict cycles (as described by [13]) that a particular reordering (a → b) participates in, it is sometimes possible to deduce the existence of another, similar error trace where a and b occur in program order, but another pair (c → d) is reordered, yielding the same conflict cycle. In such cases a fence between c and d is equally necessary as a fence between a and b.…”

“…There exists several very conservative approaches that restrict to SC executions by establishing "triangular race freedom" [12] or by inserting fences using "delay set analysis" [13]. We will not further elaborate on those techniques, but will instead focus on a number of tools and approaches more similar to our own.…”

Polynomial-Time Algorithms for Enforcing Sequential Consistency in SPMD Programs with Arrays

“…This set is called a delay set, because the second access will be delayed until the first has completed. Several researchers have proposed algorithms for finding a minimal delay set, which is the set of pairs of memory accesses whose order must be preserved in order to guarantee sequential consistency [20,11,15].…”