The Advantages of Multiple Parallelizations in Combinatorial Search

Crowl, Lawrence A.; Crovella, Mark; LeBlanc, Thomas J.; Scott, Michael L.

doi:10.1006/jpdc.1994.1045

Cited by 10 publications

(7 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…end if 14: switchToCoordinator(xc) 15: return 16: end if 17: {We win, partner will eventually register for our task} if c = ThreadRef[I] then 5: {First drop previous coordinator} 6:…”

Section: Basic Propertiesmentioning

confidence: 99%

See 1 more Smart Citation

Work-stealing for mixed-mode parallelism by deterministic team-building

Wimmer

Träff

2011

Proceedings of the Twenty-Third Annual ACM Symposium on Parallelism in Algorithms and Architectures

View full text Add to dashboard Cite

We show how to extend classical work-stealing to deal also with data parallel tasks that can require any number of threads r ≥ 1 for their execution. As threads become idle they attempt to join a team of threads designated for a task requiring r > 1 threads for its execution. Team building is done following a deterministic pattern involving log p possibly randomized steal attempts where p is the number of started hardware threads. Deterministic workstealing often exhibits good locality properties that are desirable to preserve. Threads attempting to join the team for a task requiring a large team may help smaller teams instead of waiting for the large team to form. We explain in detail the so introduced idea of work-stealing with deterministic team-building which in a natural way generalizes classical work-stealing. The implementation is done with standard lock-free data structures, in addition to which only a single extra compare-and-swap (CAS) operation per thread is required as a team is being built. Once formed, teams can stay to process further tasks requiring the same (or smaller) number of threads; this requires no further coordination. In the degenerate case, where all tasks require only a single thread, the implementation coincides with a (deterministic) work-stealing implementation, has no extra overhead, and therefore similar theoretical properties. We demonstrate correctness of the generalized work-stealing algorithm by arguing for deadlock freedom and completeness (all tasks will eventually be executed, regardless of their resource requirement r ≤ p), discuss its load-balancing, task execution order and memory-consumption properties, and discuss a number of algorithmic and implementation variations that can be considered. A prototype C++ implementation of the generalized work-stealing algorithm has been given and is briefly described. Building on this, a serious, well-known contender for a best parallel Quicksort algorithm has been implemented, which naturally relies on both task and data parallelism. On an 8-core Intel Nehalem system, a 16-core AMD Opteron system, a 16-core Sun T2+ system supporting up to * The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013.[Copyright notice will appear here once 'preprint' option is removed.] 128 hardware threads, and a 32-core Intel Nehalem EX system we compare our implementation of the published Quicksort algorithm using fork-join parallelism to a mixed-mode parallel implementation with a data parallel partitioning step using our deterministic team-building work-stealer. Results are consistently better. often by a significant fraction. For instance, sorting 2 27 − 1 randomly generated integers we could improve the speed-up from 5.1 to 8.7 on the large 32-core Intel system, on this system being consistently better than the tuned, task-parallel Cilk++ system.

show abstract

“…end if 14: switchToCoordinator(xc) 15: return 16: end if 17: {We win, partner will eventually register for our task} if c = ThreadRef[I] then 5: {First drop previous coordinator} 6:…”

Section: Basic Propertiesmentioning

confidence: 99%

“…Algorithms naturally requiring task and data parallelism and the benefit that can be expected from mixed data and task parallel programs are discussed in [5]. Centralized scheduling methods for handling mixed data and task parallel programs were discussed for instance already in [5], see also [4,6,7,9,[13][14][15].…”

Section: Introductionmentioning

confidence: 99%

Work-stealing for mixed-mode parallelism by deterministic team-building

Wimmer

Träff

2011

Proceedings of the Twenty-Third Annual ACM Symposium on Parallelism in Algorithms and Architectures

View full text Add to dashboard Cite

show abstract

“…Compiler and runtime support for task and data parallel computing is an active area of research, and several solutions have been proposed [4,5,9,10,19,21]. Recent research has also examined the benefits of mixed task and data parallel programming [3,7,18]. This paper specifically addresses the mapping of applications composed of a linear chain of data parallel tasks that act on a stream of input data sets.…”

Section: Introductionmentioning

confidence: 98%

Optimal Use of Mixed Task and Data Parallelism for Pipelined Computations

Subhlok

Vondran

2000

Journal of Parallel and Distributed Computing

View full text Add to dashboard Cite

“…Many researchers have noted that the best parallelization for a given application carl vary depending on the input, machine, or problem definition [13; 25; 27]. In fact, the best parallelization for a given application can depend on the size of the input dataset, the structure of the input dataset, the specific problem definition, the number of processors used, and the particular maa:hine used [10]. Exploring each of these environmental factors fully requires the predictive power of modeling; it is simply impractical to measure the effects of all these factors after each modification of the application.…”

Section: Introductionmentioning

confidence: 99%

Parallel performance prediction using lost cycles analysis

Crovella¹,

LeBlanc²

1994

Proceedings of the 1994 ACM/IEEE Conference on Supercomputing - Supercomputing '94

View full text Add to dashboard Cite

Most performance debugging and tuning of parallel programs is based on the \measure-modify" approach, which is heavily dependent on detailed m e asurements of programs during execution. This approach is extremely time-consuming and does not lend itself to predicting performance under varying conditions. Analytic modeling and scalability analysis provide predictive power, but are not widely used i n p r actice, due primarily to their emphasis on asymptotic behavior and the di culty of developing accurate models that work for real-world programs. In this paper we describe a s e t o f t o ols for performance tuning of parallel programs that bridges this gap between measurement and modeling.Our approach is based o n lost cycles analysis, which involves measurement and modeling of all sources of overhead in a parallel program. We rst describe a t o ol for measuring overheads in parallel programs that we have incorporated i n t o t h e runtime environment for Fortran programs on the Kendall Square KSR1. We then describe a t o ol that ts these overhead measurements to analytic forms. We illustrate the use of these tools by analyzing the performance t r adeo s among parallel implementations of 2D FFT. These examples show how our tools enable programmers to develop accurate performance m o dels of parallel applications without requiring extensive performance m o deling expertise.

show abstract

The Advantages of Multiple Parallelizations in Combinatorial Search

Cited by 10 publications

References 10 publications

Work-stealing for mixed-mode parallelism by deterministic team-building

Work-stealing for mixed-mode parallelism by deterministic team-building

Optimal Use of Mixed Task and Data Parallelism for Pipelined Computations

Parallel performance prediction using lost cycles analysis

Contact Info

Product

Resources

About