Porting irregular reductions on heterogeneous CPU-GPU configurations

Huo, Xin; Ravi, Vignesh T.; Agrawal, Gagan

doi:10.1109/hipc.2011.6152715

Cited by 18 publications

(20 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The stencil computations in MPAS often have irregular reduction modes [21] that are not suitable to be parallelized with OpenMP. This is because the input and output vectors may belong to different types of mesh points and the code may have been designed to mix arbitrary two types of unknowns.…”

Section: Regularity-aware Loop Refactoringmentioning

confidence: 99%

Pattern-Driven Hybrid Multi- and Many-Core Acceleration in the MPAS Shallow-Water Model

Zhang

Yang

et al. 2015

2015 44th International Conference on Parallel Processing

View full text Add to dashboard Cite

There is an urgent demand in studying efficient methodologies to enable hybrid multi-and many-core accelerations in global climate simulations. The Model for Prediction Across Scales (MPAS) is a family of earth-system component models that receives increasingly more attention. Like many other models, MPAS, though features some emerging numerical algorithms, employs a pure MPI approach for parallel computing, which, to date, is in lack of support for multi-threaded parallelism, especially on many-core accelerated systems. In this work, we extend the shallow-water model in MPAS to demonstrate a pattern-driven approach for hybrid multi-and many-core accelerations of climate models. We first identify all basic computation patterns through a rigorous analysis of the MPAS code. Then for the whole model, we use the identified patterns as building blocks to draw a data-flow diagram, which serves as a perfect indicator to recognize data dependencies and exploit inherent parallelism. And finally, based on the data-flow diagram, a hybrid algorithm is designed to support concurrent computations done on both multi-core CPUs and many-core accelerators. We implement the algorithm and optimize it on an x86-based heterogeneous supercomputer equipped with both Intel Xeon CPUs and Intel Xeon Phi devices. Experiments show that our hybrid design is able to deliver an 8.35x speedup as compared to the original code and scales up to 64 processes with a nearly ideal parallel efficiency.

show abstract

Section: Regularity-aware Loop Refactoringmentioning

confidence: 99%

Pattern-Driven Hybrid Multi- and Many-Core Acceleration in the MPAS Shallow-Water Model

Zhang

Yang

et al. 2015

2015 44th International Conference on Parallel Processing

View full text Add to dashboard Cite

show abstract

“…Alongside this data dependence based approach, there has also been a large body of work exploring mapping of reductions in a polyhedral setting [26,44] The treatment of more general reduction operations has received less attention. Work has focused on exploitation rather than discovery [18][19][20], examining trade-offs in implementation [52] or exploitation of novel hardware [42,51]. Recent work [16] shows that more complex reductions can be detected, but this is tied to an ad hoc non-portable code generation phase.…”

Section: Related and Future Workmentioning

confidence: 99%

Automatic Matching of Legacy Code to Heterogeneous APIs

Ginsbach

Remmelg

Steuwer

et al. 2018

Proceedings of the Twenty-Third International Conference on Architectural Support for Programming Languages and Operating Syste

View full text Add to dashboard Cite

Heterogeneous accelerators often disappoint. They provide the prospect of great performance, but only deliver it when using vendor specific optimized libraries or domain specific languages. This requires considerable legacy code modifications, hindering the adoption of heterogeneous computing.This paper develops a novel approach to automatically detect opportunities for accelerator exploitation. We focus on calculations that are well supported by established APIs: sparse and dense linear algebra, stencil codes and generalized reductions and histograms. We call them idioms and use a custom constraint-based Idiom Description Language (IDL) to discover them within user code. Detected idioms are then mapped to BLAS libraries, cuSPARSE and clSPARSE and two DSLs: Halide and Lift.We implemented the approach in LLVM and evaluated it on the NAS and Parboil sequential C/C++ benchmarks, where we detect 60 idiom instances. In those cases where idioms are a significant part of the sequential execution time, we generate code that achieves 1.26× to over 20× speedup on integrated and external GPUs.CCS Concepts • Computer systems organization → Heterogeneous (hybrid) systems; • Software and its engineering → Domain specific languages; ACM Reference Format:

show abstract

“…The appropriate utilization of hybrid systems, however, typically requires complex software instruments to deal with a number of peculiar aspects of the different processors available. This challenge has motivated a number of languages and runtime frameworks [15], [14], [16], [18], [19], [20], [21], [22], [23], [24], [25], [26], specialize libraries [4], and compiler techniques [27]. …”

Section: Related Workmentioning

confidence: 99%

“…Efficient execution of applications on distributed CPUGPU equipped platforms has been an objective of several projects [23], [24], [25], [26], [22], [29], [30]. Ravi et al [24], [26] proposes techniques for automatic translation of generalized reductions to CPU-GPU environments via compile-time techniques, which are coupled with runtime support to coordinate execution.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

High-throughput Analysis of Large Microscopy Image Datasets on CPU-GPU Cluster Platforms

Teodoro

Pan

Kurç

et al. 2013

2013 IEEE 27th International Symposium on Parallel and Distributed Processing

View full text Add to dashboard Cite

Analysis of large pathology image datasets offers significant opportunities for the investigation of disease morphology, but the resource requirements of analysis pipelines limit the scale of such studies. Motivated by a brain cancer study, we propose and evaluate a parallel image analysis application pipeline for high throughput computation of large datasets of high resolution pathology tissue images on distributed CPU-GPU platforms. To achieve efficient execution on these hybrid systems, we have built runtime support that allows us to express the cancer image analysis application as a hierarchical data processing pipeline. The application is implemented as a coarse-grain pipeline of stages, where each stage may be further partitioned into another pipeline of fine-grain operations. The fine-grain operations are efficiently managed and scheduled for computation on CPUs and GPUs using performance aware scheduling techniques along with several optimizations, including architecture aware process placement, data locality conscious task assignment, data prefetching, and asynchronous data copy. These optimizations are employed to maximize the utilization of the aggregate computing power of CPUs and GPUs and minimize data copy overheads. Our experimental evaluation shows that the cooperative use of CPUs and GPUs achieves significant improvements on top of GPU-only versions (up to 1.6×) and that the execution of the application as a set of fine-grain operations provides more opportunities for runtime optimizations and attains better performance than coarser-grain, monolithic implementations used in other works. An implementation of the cancer image analysis pipeline using the runtime support was able to process an image dataset consisting of 36,848 4Kx4K-pixel image tiles (about 1.8TB uncompressed) in less than 4 minutes (150 tiles/second) on 100 nodes of a state-of-the-art hybrid cluster system.

show abstract

Porting irregular reductions on heterogeneous CPU-GPU configurations

Cited by 18 publications

References 25 publications

Pattern-Driven Hybrid Multi- and Many-Core Acceleration in the MPAS Shallow-Water Model

Pattern-Driven Hybrid Multi- and Many-Core Acceleration in the MPAS Shallow-Water Model

Automatic Matching of Legacy Code to Heterogeneous APIs

High-throughput Analysis of Large Microscopy Image Datasets on CPU-GPU Cluster Platforms

Contact Info

Product

Resources

About