Locality Optimization of Stencil Applications Using Data Dependency Graphs

Orozco, Daniel; Garcia, Elkin; Gao, Guang R.

doi:10.1007/978-3-642-19595-2_6

Cited by 19 publications

(24 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our second study [17], we found that there is a high similarity in tasks from the same actor: They have the same properties and they execute the same function. Such similarity can be exploited to decrease the number of operations required to write those tasks in the centralized queue, greatly reducing the overhead of the runtime system.…”

Section: Runtime Systemmentioning

confidence: 70%

“…The implementation of TIDeFlow presented several challenges that ultimately resulted in interesting advances and tools: A fully distributed runtime system, a programming language to describe program graphs, concurrent algorithms [17] and new ways to reason about performance models [18].…”

Section: Implementation Of Tideflowmentioning

confidence: 99%

“…Our runtime system, with the improvements developed through our work [18,17], has resulted in a very high performance decentralized system, with overheads that are much lower than the average duration of tasks, even for fine grained programs. Section 5 shows that our runtime system is an excellent choice to support the execution of parallel programs on many-core architectures.…”

Section: Runtime Systemmentioning

confidence: 99%

“…Task management was further improved with our polytask technique [17] that took advantage of the similarity between tasks in HPC programs. The study presented in [17] shows that polytasks allow a higher efficiency when executing fine grained programs.…”

Section: Performance Of the Runtime Systemmentioning

confidence: 99%

See 3 more Smart Citations

TIDeFlow: The Time Iterated Dependency Flow Execution Model

Orozco

Garcia

Pavel

et al. 2011

2011 First Workshop on Data-Flow Execution Models for Extreme Scale Computing

Self Cite

View full text Add to dashboard Cite

The many-core revolution brought forward by recent advances in computer architecture has created immense challenges in the writing of parallel programs for High Performance Computing (HPC). Development of parallel HPC programs remains an art, and a universal doctrine for synchronization, scheduling and execution in general has not been found for many-core/multi-core architectures. These issues are exacerbated by the popularity of traditional execution models derived from the serial-programming school of thought. Previous solutions for parallel programming, such as OpenMP, MPI and similar models, require significant effort from the programmer to achieve high performance.This paper provides an introduction to the Time Iterated Dependency Flow (TIDeFlow) model, a parallel execution model inspired by dataflow, and a description of its associated runtime system. TIDeFlow was designed for efficient development of high performance parallel programs for many-core architectures.The TIDeFlow execution model was designed to efficiently express (1) parallel loops, (2) dependencies (data, control or other) between parallel loops and (3) to allow composability of programs.TIDeFlow is a work in progress. This paper presents an introduction to the TIDeFlow execution model and shows examples and preliminary results to illustrate the qualities of TIDeFlow.The main contributions of this paper are:1. A brief description of the TIDeFlow execution model, and its programming model, 2. A description of the implementation of the TIDeFlow runtime system and its capabilities and 3. Preliminary results showing the suitability of TIDeFlow to express parallel programs in many-core archiPermission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

show abstract

Section: Runtime Systemmentioning

confidence: 70%

Section: Implementation Of Tideflowmentioning

confidence: 99%

Section: Runtime Systemmentioning

confidence: 99%

Section: Performance Of the Runtime Systemmentioning

confidence: 99%

See 2 more Smart Citations

TIDeFlow: The Time Iterated Dependency Flow Execution Model

Orozco

Garcia

Pavel

et al. 2011

2011 First Workshop on Data-Flow Execution Models for Extreme Scale Computing

Self Cite

View full text Add to dashboard Cite

show abstract

“…Several applications were tested: Fast Fourier Transforms that use the CooleyTukey algorithm with two-point butterflies (FFT) and simulations of electromagnetic waves propagating using the Finite Difference Time Domain algorithm in 1 Dimension (FDTD1D) [15] and 2 dimensions (FDTD2D) [14]. FFT was run with input sizes 2 9 (FFT2P 2 9 ) through 2 12 (FFT2P 2 12 ), FDTD1D runs a problem of size 20000 with 3 timesteps and tiles of width 16, and FDTD2D runs a problem of size 128 by 128 with 2 timesteps and tiles of width 4 by 4.…”

Section: Applicationsmentioning

confidence: 99%

Polytasks: A Compressed Task Representation for HPC Runtimes

Orozco

Garcia

Pavel

et al. 2013

Languages and Compilers for Parallel Computing

Self Cite

View full text Add to dashboard Cite

Abstract. The increased number of execution units in many-core processors is driving numerous paradigm changes in parallel systems. Previous techniques that focused solely upon obtaining correct results are being rendered obsolete unless they can also provide results efficiently. This paper dives into the particular problem of efficiently supporting fine-grained task creation and task termination for runtime systems in shared memory processors. Our contributions are inspired by our observation of High Performance Computing (HPC) programs, where it is common for a large number of similar fine-grained tasks to become enabled at the same time. We present evidence showing that task creation, assignment of tasks to processors, and task termination represent a significant overhead when executing fine-grained applications in many-core processors. We introduce the concept of the polytask, wherein the similarity of tasks created at the same time is exploited to allow faster task creation, assignment and termination. The polytask technique can be applied to any runtime system where tasks are managed through queues. The main contributions of this work are:1. The observation that task management may generate substantial overhead in fine-grained parallel programs for many core processors. 2. The introduction of the polytask concept: A data structure that can be added to queue-centric scheduling systems to represent groups of similar tasks. 3. Experimental evidence showing that the polytask is an effective way to implement fine-grained task creation/termination primitives for parallel runtime systems in many-core processors. We use microbenchmarks to show that queues modified to handle polytasks perform orders of magnitude faster than traditional queues in some scenarios. Furthermore, we use microbenchmarks to measure the amount of time spent executing tasks. We show situations where fine-grained programs using polytasks are able to achieve efficiencies close to 100% while their efficiency becomes only 20% when not using polytasks. Finally, we use several applications with fine granularity to show that the use of polytasks results in average speedups from 1.4X to 100X depending on the queue implementation used.

show abstract

TOAST: Automatic tiling for iterative stencil computations on GPUs

Rocha

Pereira

Ramos

et al. 2017

Concurrency and Computation

View full text Add to dashboard Cite

Summary The stencil pattern is important in many scientific and engineering domains, spurring great interest from researchers and industry. In recent years, various optimizations have been proposed for parallel stencil applications running on graphics processing units (GPUs). In particular, tiling is a technique that can significantly enhance application performance by improving data locality and by reducing the volume of communication between host memory and GPU. In addition, tiling enables stencil applications to process inputs that are larger than the physical GPU memory. However, implementing tiling efficiently is complex, time‐consuming, and error‐prone. In this paper, we propose transparently optimized automatic stencil tiling (TOAST), an automatic tiling mechanism for iterative stencil computations running on GPUs; TOAST has 3 main benefits: (1) It incorporates an optimization model that seeks to maximize data reuse within tiles while respecting the amount of dynamically available GPU memory; (2) it offers a virtualized GPU memory for stencil computations, allowing for large input data; and (3) it performs optimal tiling transparently to the developer of the parallel stencil application. The current implementation of TOAST augments the PSkel framework with an internal solver based on genetic algorithms. Our experimental results show that TOAST improves the performance of iterative stencil applications by up to 13 × compared with their multithreaded (central processing unit–based) optimized versions and up to 48 × compared with a naive tiling approach on GPU. The TOAST mechanism is able to automatically achieve a low percentual overhead of data management compared with actual stencil computation.

show abstract

Locality Optimization of Stencil Applications Using Data Dependency Graphs

Cited by 19 publications

References 13 publications

TIDeFlow: The Time Iterated Dependency Flow Execution Model

TIDeFlow: The Time Iterated Dependency Flow Execution Model

Polytasks: A Compressed Task Representation for HPC Runtimes

TOAST: Automatic tiling for iterative stencil computations on GPUs

Contact Info

Product

Resources

About