Parallel Algorithms for Asymmetric Read-Write Costs

Abstract

Motivated by the significantly higher cost of writing than reading in emerging memory technologies, we consider parallel algorithm design under such asymmetric read-write costs, with the goal of reducing the number of writes while preserving work-efficiency and low span. We present a nested-parallel model of computation that combines (i) small per-task stack-allocated memories with symmetric read-write costs and (ii) an unbounded heap-allocated shared memory with asymmetric read-write costs, and show how the costs in the model map efficiently onto a more concrete machine model under a work-stealing scheduler. We use the new model to design reduced-write, work-efficient, low-span parallel algorithms for a number of fundamental problems such as reduce, list contraction, tree contraction, breadth-first search, ordered filter, and planar convex hull. For the latter two problems, our algorithms are output-sensitive in that the work and number of writes decrease with the output size. We also present a reduced-write, low-span minimum spanning tree algorithm that is nearly work-efficient (off by the inverse Ackermann function). Our algorithms reveal several interesting techniques for significantly reducing shared memory writes in parallel algorithms without asymptotically increasing the number of shared memory reads.

Improved Parallel Cache-Oblivious Algorithms for Dynamic Programming [Extend Abstract]

“…This model captures different system consideration (latency, bandwidth, or energy) by simply plugging in a different value of ω, and also allows algorithms to be analyzed theoretically and practically. Similar scheduling results (upper bounds) on parallel running time and cache complexity are discussed in [13,17] based on work W , span D and asymmetric cache complexity Q of an algorithm. Based on this idea, many interesting algorithms and lower bounds are designed and analyzed by various recent works [13,16,17,23,61,14,21,19,53].…”

“…The challenge arises in scheduling this computation. The scheduling theorem for the asymmetric case [13] constraints on the non-leaf stack memory to be a constant size. This contradicts the parallel version in Section 6.1.…”

“…Then all successive heap-based memory allocation can be simulated on this chunk of memory. In this manner, the stack memory of each node corresponding to a function call is constant, which allows us to apply the scheduling theorem in [13]. Theorem 6.8.…”

Implicit Decomposition for Write-Efficient Connectivity Algorithms

“…8 If the search exhausts all vertices with center v, return. 9 Otherwise identify a vertex u that partitions the tree such that its subtree and the rest of the tree are each at least a constant fraction of k.…”

“…The key subroutine of the algorithm, however, is just breadth-first searches (BFS's). Replacing these BFS's by write-efficient BFS's [9] yields the following theorem:…”