Sorting, Searching, and Simulation in the MapReduce Framework

Abstract

In this paper, we study the MapReduce framework from an algorithmic standpoint and demonstrate the usefulness of our approach by designing and analyzing efficient MapReduce algorithms for fundamental sorting, searching, and simulation problems. This study is motivated by a goal of ultimately putting the MapReduce framework on an equal theoretical footing with the well-known PRAM and BSP parallel models, which would benefit both the theory and practice of MapReduce algorithms. We describe efficient MapReduce algorithms for sorting, multi-searching, and simulations of parallel algorithms specified in the BSP and CRCW PRAM models. We also provide some applications of these results to problems in parallel computational geometry for the MapReduce framework, which result in efficient MapReduce algorithms for sorting, 2-and 3-dimensional convex hulls, and fixed-dimensional linear programming. For the case when mappers and reducers have a memory/message-I/O size of M = Θ(N ), for a small constant > 0, all of our MapReduce algorithms for these applications run in a constant number of rounds.

On the Computational Complexity of MapReduce

“…[8]. The MRC class of Karloff et al is the closest to existing MapReduce computations, and is also among the most restrictive in terms of how it handles communication and tracks the computational power of individual processors.…”

“…Kamara and Raykova [11] develop a homomorphic encryption scheme for MapReduce. And much work has been done on graph problems such as connectivity, matchings, sorting, and searching [8]. Chu et al [3] demonstrate the potential to express any statistical-query learning algorithm in MapReduce.…”

“…Goodrich et al [8] and Pace [15] showed that MapReduce computations can be simulated in the BSP model and vice versa, with only a constant blow-up in the computational resources needed. This implies that our theorems about MapReduce automatically apply to BSP.…”

An approach towards the study of symmetric queries

“…Retrieve the pairs of products that occur together in at least two transactions. 7. Retrieve the pairs of students taking the same courses.…”

“…A good example of this is MapReduce, where it is commonly assumed that the reducer and combiner functions are symmetric [5,6,7,9]. Typically, it is up to the programmer to guarantee that this property is satisfied.…”