Parallelizing complex scans and reductions

Fisher, Allan L.; Ghuloum, Anwar

doi:10.1145/178243.178255

Cited by 61 publications

(24 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Discovering and exploiting scalar reductions in programs has been studied for many years based on dependence analysis and idiom detection [13,27,34]. Early work focused on well structured Fortran and paid little attention to automatic compiler-based detection, a notable exception being [34].…”

Section: Related Workmentioning

confidence: 99%

Discovery and exploitation of general reductions: A constraint based approach

Ginsbach¹,

O’Boyle²

2017

2017 IEEE/ACM International Symposium on Code Generation and Optimization (CGO)

View full text Add to dashboard Cite

Discovering and exploiting scalar reductions in programs has been studied for many years. The discovery of more complex reduction operations has, however, received less attention. Such reductions contain compile-time unknown parameters, indirect memory accesses and dynamic control flow, which are challenging for existing approaches.In this paper we develop a new compiler based approach that automatically detects a wide class of reductions. This is based on a constraint formulation of the reduction idiom and has been implemented as an LLVM pass. We use a custom constraint solver to identify program subsets that adhere to the constraint specification. Once discovered, we automatically generate parallel code to exploit the reduction.This approach is robust and was evaluated on C versions of well known benchmark suites: NAS, Parboil and Rodinia. We detected 84 scalar reductions and 6 histograms, outperforming existing approaches. We show that the exploitation of histograms gives significant performance improvement.

show abstract

Section: Related Workmentioning

confidence: 99%

Discovery and exploitation of general reductions: A constraint based approach

Ginsbach¹,

O’Boyle²

2017

2017 IEEE/ACM International Symposium on Code Generation and Optimization (CGO)

View full text Add to dashboard Cite

show abstract

“…We show that a weak right inverse always exists and can be automatically generated for a wide class of sequential functions. As will be seen later, this new approach is applicable to many nontrivial examples, including the maximum prefix sum problem [6,14,15], the prefix sum computation [6], the maximum segment sum problem [3], and the line-of-sight problem [6].…”

Section: Mps ([A] + + Y) = 0 ↑ (A + Mps(y))mentioning

confidence: 99%

Automatic inversion generates divide-and-conquer parallel programs

Morita

Morihata

Matsuzaki

et al. 2007

Proceedings of the 28th ACM SIGPLAN Conference on Programming Language Design and Implementation

View full text Add to dashboard Cite

Divide-and-conquer algorithms are suitable for modern parallel machines, tending to have large amounts of inherent parallelism and working well with caches and deep memory hierarchies. Among others, list homomorphisms are a class of recursive functions on lists, which match very well with the divide-and-conquer paradigm. However, direct programming with list homomorphisms is a challenge for many programmers. In this paper, we propose and implement a novel system that can automatically derive costoptimal list homomorphisms from a pair of sequential programs, based on the third homomorphism theorem. Our idea is to reduce extraction of list homomorphisms to derivation of weak right inverses. We show that a weak right inverse always exists and can be automatically generated from a wide class of sequential programs. We demonstrate our system with several nontrivial examples, including the maximum prefix sum problem, the prefix sum computation, the maximum segment sum problem, and the line-ofsight problem. The experimental results show practical efficiency of our automatic parallelization algorithm and good speedups of the generated parallel programs.

show abstract

“…In Bellmania, more focus is put on re-ordering of array reads and writes, following and mechanizing techniques related to DP from [8,9]. In fact, traditional parallelism is taking "for granted" for our aggregation operators, since they are associative and methods such as [17] apply rather trivially, and translated into the tactics Slice, Assoc, and Distrib. On top of these algebraic transformations, Bellmania allows clever re-orderings, especially through Stratify and Let.…”

Section: Related Workmentioning

confidence: 99%

Deriving divide-and-conquer dynamic programming algorithms using solver-aided transformations

Itzhaky

Singh

Solar-Lezama

et al. 2016

Proceedings of the 2016 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applicatio

View full text Add to dashboard Cite

We introduce a framework allowing domain experts to manipulate computational terms in the interest of deriving better, more efficient implementations. It employs deductive reasoning to generate provably correct efficient implementations from a very high-level specification of an algorithm, and inductive constraint-based synthesis to improve automation. Semantic information is encoded into program terms through the use of refinement types.In this paper, we develop the technique in the context of a system called Bellmania that uses solver-aided tactics to derive parallel divide-and-conquer implementations of dynamic programming algorithms that have better locality and are significantly more efficient than traditional loop-based implementations. Bellmania includes a high-level language for specifying dynamic programming algorithms and a calculus that facilitates gradual transformation of these specifications into efficient implementations. These transformations formalize the divide-and-conquer technique; a visualization interface helps users to interactively guide the process, while an SMT-based back-end verifies each step and takes care of low-level reasoning required for parallelism.We have used the system to generate provably correct implementations of several algorithms, including some important algorithms from computational biology, and show that the performance is comparable to that of the best manually optimized code.

show abstract

Parallelizing complex scans and reductions

Cited by 61 publications

References 15 publications

Discovery and exploitation of general reductions: A constraint based approach

Discovery and exploitation of general reductions: A constraint based approach

Automatic inversion generates divide-and-conquer parallel programs

Deriving divide-and-conquer dynamic programming algorithms using solver-aided transformations

Contact Info

Product

Resources

About