Supporting Irregularity in Throughput-Oriented Computing by SIMT-SIMD Integration

Thuerck, Daniel

doi:10.1109/ia351965.2020.00010

Cited by 2 publications

(1 citation statement)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thuerck [238] proposed a hardware and compilersupported extension to CUDA's programming that handles data and control-flow irregularities. Major architectural enhancements were implemented and tested in a simulation environment.…”

Section: ) Hardware Supported Modifications (Hardware Solution)mentioning

confidence: 99%

MIMD Programs Execution Support on SIMD Machines: A Holistic Survey

Mustafa,

Alkhasawneh,

Obeidat

et al. 2024

IEEE Access

View full text Add to dashboard Cite

The Single Instruction Multiple Data (SIMD) architecture, supported by various highperformance computing platforms, efficiently utilizes data-level parallelism. The SIMD model is used in traditional CPUs, dedicated vector systems, and accelerators such as GPUs, vector extensions, and Xeon Phi. It provides performance throughput in computation-intensive and data-parallel applications. Despite the similarity of data-processing principles between these architectures, porting various programming models between the reviewed platforms is challenging. Furthermore, enhancing the programmability of these architectures is an important feature for utilizing their emerging computing power and simplifying programming complexity. This paper reviews the basic principles of optimization techniques to run asynchronous Multiple Instruction Multiple Data (MIMD) on SIMD accelerators. It also surveys several GPU programming paradigms and application programming interfaces (APIs) and classifies these frameworks into different groups based on their criteria. In addition, a review of studies that performed a comparison of the collaborative execution of GPUs with CPUs and Xeon Phi is presented in this paper. This study will be beneficial for developers and researchers in the field of computer architecture and parallel computing of intensive scientific applications, specifically for early-stage high-performance computing researchers, to obtain a brief overview of performance optimization opportunities as well as the challenges of existing SIMD platforms.

show abstract

Section: ) Hardware Supported Modifications (Hardware Solution)mentioning

confidence: 99%

MIMD Programs Execution Support on SIMD Machines: A Holistic Survey

Mustafa,

Alkhasawneh,

Obeidat

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

Flynn’s Reconciliation

Thuerck

Weber²,

Bifulco³

2021

ACM Trans. Archit. Code Optim.

Self Cite

View full text Add to dashboard Cite

A large portion of the recent performance increase in the High Performance Computing (HPC) and Machine Learning (ML) domains is fueled by accelerator cards. Many popular ML frameworks support accelerators by organizing computations as a computational graph over a set of highly optimized, batched general-purpose kernels. While this approach simplifies the kernels’ implementation for each individual accelerator, the increasing heterogeneity among accelerator architectures for HPC complicates the creation of portable and extensible libraries of such kernels. Therefore, using a generalization of the CUDA community’s warp register cache programming idiom, we propose a new programming idiom (CoRe) and a virtual architecture model (PIRCH), abstracting over SIMD and SIMT paradigms. We define and automate the mapping process from a single source to PIRCH’s intermediate representation and develop backends that issue code for three different architectures: Intel AVX512, NVIDIA GPUs, and NEC SX-Aurora. Code generated by our source-to-source compiler for batched kernels, borG, competes favorably with vendor-tuned libraries and is up to 2× faster than hand-tuned kernels across architectures.

show abstract

Supporting Irregularity in Throughput-Oriented Computing by SIMT-SIMD Integration

Cited by 2 publications

References 17 publications

MIMD Programs Execution Support on SIMD Machines: A Holistic Survey

MIMD Programs Execution Support on SIMD Machines: A Holistic Survey

Flynn’s Reconciliation

Contact Info

Product

Resources

About