External Sorting and Permuting

Vitter, Jeffrey Scott

doi:10.1007/978-0-387-30162-4_137

Cited by 6 publications

(6 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This approach therefore can characterize the limits of parallelism that are possible in a MapReduce algorithm and it also shows that we should concentrate on the round complexity and communication complexity of a MapReduce algorithm in characterizing its performance 3 . Of course, such bounds for R and C may depend on M , but that is fine, for similar characterizations are common in the literature on external-memory algorithms (e.g., see [1,3,4,18,19]). In the rest of the paper, when we talk about the MapReduce model, we always mean the I/O-memory-bound MapReduce model except when mentioned explicitly.…”

Section: Memory-bound and I/o-bound Mapreduce Algorithmsmentioning

confidence: 95%

Sorting, Searching, and Simulation in the MapReduce Framework

Goodrich

Sitchinava

Zhang

2011

Lecture Notes in Computer Science

155

116

View full text Add to dashboard Cite

In this paper, we study the MapReduce framework from an algorithmic standpoint and demonstrate the usefulness of our approach by designing and analyzing efficient MapReduce algorithms for fundamental sorting, searching, and simulation problems. This study is motivated by a goal of ultimately putting the MapReduce framework on an equal theoretical footing with the well-known PRAM and BSP parallel models, which would benefit both the theory and practice of MapReduce algorithms. We describe efficient MapReduce algorithms for sorting, multi-searching, and simulations of parallel algorithms specified in the BSP and CRCW PRAM models. We also provide some applications of these results to problems in parallel computational geometry for the MapReduce framework, which result in efficient MapReduce algorithms for sorting, 2-and 3-dimensional convex hulls, and fixed-dimensional linear programming. For the case when mappers and reducers have a memory/message-I/O size of M = Θ(N ), for a small constant > 0, all of our MapReduce algorithms for these applications run in a constant number of rounds.

show abstract

Section: Memory-bound and I/o-bound Mapreduce Algorithmsmentioning

confidence: 95%

Sorting, Searching, and Simulation in the MapReduce Framework

Goodrich

Sitchinava

Zhang

2011

Lecture Notes in Computer Science

155

116

View full text Add to dashboard Cite

show abstract

“…Thus, we can simply refer to the memory and message sizes in terms of the number of items that are stored. This notation is borrowed from the literature on external-memory algorithms (e.g., see [21]), since it closely models the scenario where the memory needed by a computation exceeds its local capacity so that external storage is needed. In keeping with this analogy to external-memory algorithms, we refer to each message that is exchanged between Alice and Bob as an I/O, each of which, as noted above, is of size at most M .…”

Section: Our Resultsmentioning

confidence: 99%

Practical oblivious storage

Goodrich

Mitzenmacher

Ohrimenko

et al. 2012

Proceedings of the Second ACM Conference on Data and Application Security and Privacy

View full text Add to dashboard Cite

We study oblivious storage (OS), a natural way to model privacy-preserving data outsourcing where a client, Alice, stores sensitive data at an honest-but-curious server, Bob. We show that Alice can hide both the content of her data and the pattern in which she accesses her data, with high probability, using a method that achieves O(1) amortized rounds of communication between her and Bob for each data access. We assume that Alice and Bob exchange small messages, of size O(N 1/c ), for some constant c ≥ 2, in a single round, where N is the size of the data set that Alice is storing with Bob. We also assume that Alice has a private memory of size 2N 1/c . These assumptions model real-world cloud storage scenarios, where trade-offs occur between latency, bandwidth, and the size of the client's private memory.

show abstract

“…In this paper we describe efficient external-memory implementations of the multimap ADT. Our externalmemory algorithms are for the standard two-level I/O model, which captures the memory hierarchy of modern computer architectures (e.g., see [1,22]). In this model, there is a cache of size M connected to a disk of unbounded size, and the cache and disk are divided into blocks, where each block can store up to B items.…”

Section: Our Resultsmentioning

confidence: 99%

External-Memory Multimaps

et al. 2013

View full text Add to dashboard Cite

Many data structures support dictionaries, also known as maps or associative arrays, which store and manage a set of key-value pairs. A multimap is generalization that allows multiple values to be associated with the same key. For example, the inverted file data structure that is used prevalently in the infrastructure supporting search engines is a type of multimap, where words are used as keys and document pointers are used as values. We study the multimap abstract data type and how it can be implemented efficiently online in external memory frameworks, with constant expected I/O performance.The key technique used to achieve our results is a combination of cuckoo hashing using buckets that hold multiple items with a multiqueue implementation to cope with varying numbers of values per key. Our external-memory results are for the standard two-level memory model.

show abstract

External Sorting and Permuting

Cited by 6 publications

References 11 publications

Sorting, Searching, and Simulation in the MapReduce Framework

Sorting, Searching, and Simulation in the MapReduce Framework

Practical oblivious storage

External-Memory Multimaps

Contact Info

Product

Resources

About