An analysis of dag-consistent distributed shared-memory algorithms

Abstract

In this paper, we analyze the performance of parallel multithreaded algorithms that use dag-consistent distributed shared memory. Specifically, we analyze execution time, page faults, and space requirements for multithreaded algorithms executed by a workstealing thread scheduler and the BACKER algorithm for maintaining dag consistency. We prove that if the accesses to the backing store are random and independent (the BACKER algorithm actually uses hashing), the expected execution time T P (C) of a "fully strict" multithreaded computation on P processors, each with a LRU cache of C pages, is O(T 1 (C)=P + mCT ∞ ), where T 1 (C) is the total work of the computation including page faults, T ∞ is its critical-path length excluding page faults, and m is the minimum page transfer time. As a corollary to this theorem, we show that the expected number F P (C) of page faults incurred by a computation executed on P processors can be related to the number F 1 (C) of serial page faults by the formula F P (C) F 1 (C) + O(CPT ∞ ). Finally, we give simple bounds on the number of page faults and the space requirements for "regular" divide-and-conquer algorithms. We use these bounds to analyze parallel multithreaded algorithms for matrix multiplication and LU-decomposition.

The Cache Complexity of Multithreaded Cache Oblivious Algorithms

“…Whether this assumption is true in practice depends on the consistency model maintained by the caches. For example, caches are noninterfering in the dag-consistent memory model maintained by the Backer protocol [7]. Alternatively, caches are noninterfering in the HSMS model [1] if the computation is race-free.…”

“…Our ideal distributed cache model is almost the same as the dag-consistent model analyzed by Blumofe et al [7], except that we assume ideal caches instead of caches with LRU replacement. Bender et al [3] consider a distributedcache model, but with cache coherence and atomic operations.…”

“…For series-parallel dags, however, more encouraging results are known. Blumofe et al [7] prove that the Cilk randomized work-stealing scheduler [9] executes a series-parallel computation on P processors incurring…”

“…For example, a multithreaded program for multiplying two n × n matrices without using additional storage has work T 1 = O(n 3 ), critical path T ∞ = O(n), and sequential cache complexity Q 1 = O(n 3 / √ Z + n 2 ) [7]. When the program is executed on P processors by the Cilk scheduler, we prove that its cache complexity is Q P = O n 3 / √ Z + (T ∞ P ) 1/3 n 2 with high probability.…”

Efficient Detection of Determinacy Races in Cilk Programs

“…Nevertheless, we feel that it may be possible to develop a parallel version of the SP-bags algorithm. We have started investigating a parallel scheme in which each of several processors executing the program uses the SP-bags algorithm locally, but when a remote child procedure returns, it reconciles its shadow spaces in a manner similar to the BACKER algorithm [4] for maintaining dag consistency. Such a result may be mostly of theoretical interest, however, since debugging is usually done in the development phase of a program using small data sets, and thus typically, the performance of the debugger is not a crucial concern.…”