Resource Oblivious Sorting on Multicores

Cole, Richard; Ramachandran, Vijaya

doi:10.1145/3040221

Cited by 28 publications

(59 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Notice that ω is a parameter of the main memory, instead of a cache parameter, so the algorithms can be aware of it. One can define resource-obliviousness [37] so that the value of ω is not exposed to the algorithms, but this is out of the scope of this paper.…”

Section: Asymmetric Cache Complexitymentioning

confidence: 99%

Improved Parallel Cache-Oblivious Algorithms for Dynamic Programming [Extend Abstract]

Blelloch¹,

Gu²

2020

Symposium on Algorithmic Principles of Computer Systems

View full text Add to dashboard Cite

Emerging non-volatile main memory (NVRAM) technologies provide byte-addressability, low idle power, and improved memory-density, and are likely to be a key component in the future memory hierarchy. However, a critical challenge in achieving high performance is in accounting for the asymmetry that NVRAM writes can be significantly more expensive than NVRAM reads.In this paper, we consider a large class of cache-oblivious algorithms for dynamic programming (DP) and linear algebra, and try to reduce the writes in the asymmetric setting while maintaining high parallelism. To achieve that, our key approach is to show the correspondence between these problems and an abstraction for their computation, which is referred to as the k-d grids. Then by showing lower bound and new algorithms for computing k-d grids, we show a list of improved cache-oblivious algorithms of many DP recurrences and in linear algebra in the asymmetric setting, both sequentially and in parallel.Surprisingly, even without considering the read-write asymmetry (i.e., setting the write cost to be the same as the read cost in the algorithms), the new algorithms improve the existing cache complexity of many problems. We believe the reason is that the extra level of abstraction of k-d grids helps us to better understand the complexity and difficulties of these problems. We believe that the novelty of our framework is of interests and leads to many new questions for future work.

show abstract

Section: Asymmetric Cache Complexitymentioning

confidence: 99%

Improved Parallel Cache-Oblivious Algorithms for Dynamic Programming [Extend Abstract]

Blelloch¹,

Gu²

2020

Symposium on Algorithmic Principles of Computer Systems

View full text Add to dashboard Cite

show abstract

“…By applying the analysis in [14] with the change that the base cases (for the recursive sort and the transpose) are when the size fits in the ephemeral memory, and that the base case is done sequentially, we obtain the following theorem. It is possible that the log n term in the depth could be reduced using a sort by Cole and Ramachandran [23].…”

Section: Fault-tolerant Algorithmsmentioning

confidence: 99%

The Parallel Persistent Memory Model

Blelloch

Gibbons

et al. 2018

Proceedings of the 30th on Symposium on Parallelism in Algorithms and Architectures

View full text Add to dashboard Cite

We consider a parallel computational model, the Parallel Persistent Memory model, comprised of P processors, each with a fast local ephemeral memory of limited size, and sharing a large persistent memory. The model allows for each processor to fault at any time (with bounded probability), and possibly restart. When a processor faults, all of its state and local ephemeral memory is lost, but the persistent memory remains. This model is motivated by upcoming non-volatile memories that are nearly as fast as existing random access memory, are accessible at the granularity of cache lines, and have the capability of surviving power outages. It is further motivated by the observation that in large parallel systems, failure of processors and their caches is not unusual.We present several results for the model, using an approach that breaks a computation into capsules, each of which can be safely run multiple times. For the single-processor version we describe how to simulate any program in the RAM, the external memory model, or the ideal cache model with an expected constant factor overhead. For the multiprocessor version we describe how to efficiently implement a work-stealing scheduler within the model such that it handles both soft faults, with a processor restarting, and hard faults, with a processor permanently failing. For any multithreaded fork-join computation that is race free, write-after-read conflict free and has W work, D depth, and C maximum capsule work in the absence of faults, the scheduler guarantees a time bound on the model of O W P A + DP P A log 1/(Cf ) W in expectation, where P is the maximum number of processors, P A is the average number, and f ≤ 1/(2C) is the probability a processor faults between successive persistent memory accesses. Within the model, and using the proposed methods, we develop efficient algorithms for parallel prefix sums, merging, sorting, and matrix multiply. * This paper is the full version of a paper at SPAA 2018 with the same name.

show abstract

“…Although not explicitly stated, this observation appears to need the assumption that all variables have a fixed allocation, regardless of the amount of parallelism. In [17], we showed how to account for dynamically allocated variables (the issue being that block boundaries on the execution stack for a stolen task can diverge from those at the parent task), giving a bound of O(Q + S · M/B) on C(S). Frigo and Strumpen [20] considered the above set-up for computations where any fragment of size r that occurs in a parallel execution incurs O(f (r)) cache misses, for some concave function f .…”

Section: Related Workmentioning

confidence: 99%

“…Another reason for considering a general scheduler is to obtain 'oblivious' results as in sequential cache-oblivious algorithms [19], network-oblivious algorithms for distributed memory [5], and multicore-oblivious [13] and resource-oblivious [17,15] algorithms for shared memory multicores. In all of these cases the desire is to have algorithms analyzed in a machine-independent manner so that bounds hold across diverse platforms.…”

Section: Related Workmentioning

confidence: 99%

“…Our bounds match the best bounds known for work stealing schedulers. The class of algorithms we consider includes efficient multithreaded algorithms for several fundamental problems such as matrix multiplication [19], the Gaussian Elimination Paradigm (GEP) [12], longest common subsequence (LCS) and related dynamic programming problems [12,10], FFT [19], SPMS sorting [17], list ranking [13], and graph connectivity [13]. These are all well-known multithreaded algorithms that use parallel recursive divide and conquer.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Bounding Cache Miss Costs of Multithreaded Computations Under General Schedulers

Cole

Ramachandran

2017

Proceedings of the 29th ACM Symposium on Parallelism in Algorithms and Architectures

Self Cite

View full text Add to dashboard Cite

We analyze the caching overhead incurred by a class of multithreaded algorithms when scheduled by an arbitrary scheduler. We obtain bounds that, modulo constant factors, match or improve the well-known O(Q + S · (M/B)) caching cost for the randomized work stealing (RWS) scheduler, where S is the number of steals, Q is the sequential caching cost, and M and B are the cache size and block (or cache line) size respectively.

show abstract

Resource Oblivious Sorting on Multicores

Cited by 28 publications

References 30 publications

Improved Parallel Cache-Oblivious Algorithms for Dynamic Programming [Extend Abstract]

Improved Parallel Cache-Oblivious Algorithms for Dynamic Programming [Extend Abstract]

The Parallel Persistent Memory Model

Bounding Cache Miss Costs of Multithreaded Computations Under General Schedulers

Contact Info

Product

Resources

About