A provably time-efficient parallel implementation of full speculation

Abstract

Speculative evaluation, including leniency and futures, is often used to produce high degrees of parallelism. Understanding the performance characteristics of such evaluation, however, requires having a detailed understanding of the implementation. For example, the particular implementation technique used to suspend and reactivate threads can have an asymptotic effect on performance. With the goal of giving the users some understanding of performance without requiring them to understand the implementation, we present a provable implementation bound for a language based on speculative evaluation. The idea is (1) to supply the users with a semantics for a language that defines abstract costs for measuring or analyzing the performance of computations, (2) to supply the users with a mapping of these costs onto runtimes on various machine models, and (3) to describe an implementation strategy of the language and prove that it meets these mappings. For this purpose we consider a simple language based on speculative evaluation. For every computation, the semantics of the language returns a directed acyclic graph (DAG) in which each node represents a unit of computation, and each edge represents a dependence. We then describe an implementation strategy of the language and show that any computation with w work (the number of nodes in the DAG) and d depth (the length of the longest path in the DAG) will run on a p-processor PRAM in O(w/p + d log p) time. The bounds are work efficient (within a constant factor of linear speedup) when there is sufficient parallelism, w/d ≥ p log p. These are the first time bounds we know of for languages with speculative evaluation. The main challenge is in parallelizing the necessary queuing operations on suspended threads.

Cache efficient functional algorithms

“…The present work extends the methodology of Blelloch and Greiner [4,13] to account for the cache complexity of algorithms stated in terms of two parameters, the cache block size and the number of cache blocks. Analyses are carried out in terms of the semantics of ICFP given in Section 2, and are transferred to the Ideal Cache Model by a provable implementation, also sketched in Section 2.…”

“…Based on previous work [4,13] we expect that the evaluation semantics given here will provide a good foundation for specifying parallel as well as sequential complexity. One complication is that the explicit consideration of storage considerations in the cost model given here would have to take account of the interaction among parallel threads.…”

“…The general idea of using high-level cost models based on a cost semantics along with a provable efficient implementation has previously been used in the context of parallel cost models [4,13,22,15]. Although there has been a large amount of experimental work on showing how good garbage collection can lead to efficient use of caches and disks ( [10,24,14,7] and many references in [16]), we know of none that try to prove bounds for algorithms for functional programs when manipulating recursive data types such as lists or trees.…”

“…The abstract cost semantics underlies the analyses given in Sections 1 and 3. The abstract costs given by the semantics are validated by giving a provable implementation [4,13] that realizes these costs on a concrete machine. In our case the realization is given in terms of the ICMM, which is an accepted concrete formulation of a machine with hierarchical memory.…”

From sequential programs to multi-tier applications by program transformation

“…Research on the parallel implementation of functional languages is concerned with the automatic detection of implicit parallelism and the speculative evaluation of expressions, e.g., [13,4]. Our analysis does not detect parallelism but determines independent slices of programs with explicit communication interfaces.…”