Fundamental parallel algorithms for private-cache chip multiprocessors

Abstract

In this paper, we study parallel algorithms for private-cache chip multiprocessors (CMPs), focusing on methods for foundational problems that can scale to hundreds or even thousands of cores. By focusing on private-cache CMPs, we show that we can design efficient algorithms that need no additional assumptions about the way that cores are interconnected, for we assume that all inter-processor communication occurs through the memory hierarchy. We study several fundamental problems, including prefix sums, selection, and sorting, which often form the building blocks of other parallel algorithms. Indeed, we present two sorting algorithms, a distribution sort and a mergesort. All algorithms in the paper are asymptotically optimal in terms of the parallel cache accesses and space complexity under reasonable assumptions about the relationships between the number of processors, the size of memory, and the size of cache blocks. In addition, we study sorting lower bounds in a computational model, which we call the parallel external-memory (PEM) model, that formalizes the essential properties of our algorithms for private-cache chip multiprocessors.

On (Dynamic) Range Minimum Queries in External Memory

“…Our solution also implies a linear space parallel I/O-efficient solution to the RMQ problem in the parallel external memory (PEM) model [3]. …”

“…The algorithm of Chiang et al [7] has been used by Arge et al [4] to develop a parallel solution to the static batched RMQ problem in the parallel external memory (PEM) model [3]. Our new linear space EM solution immediately leads to a linear space PEM solution.…”

“…Note that x i is not the number of elements currently stored in the leaves of the subtree rooted at w i , because there could still be some inserts and/or deletes waiting in the buffers between w i and the leaves of the subtree rooted at w i . 3 Hence, during the buffer emptying process, we can route the operations to the correct children when emptying a buffer: at the root v with children w 1 , . .…”

Parallel external memory graph algorithms

“…In contrast to the cache-oblivious multicore-cache model, Arge et al [3] proposed a cache-aware parallel external memory (PEM) model. The authors provide several basic parallel techniques and solutions to fundamental combinatorial problems, such as prefix sums and sorting.…”

“…Recently Arge et al [3] introduced the parallel external memory (PEM) model to model the modern CMP architectures. The PEM model is a parallelization of the sequential external memory (EM) model of Aggarwal and Vitter [1].…”