Enabling One-size-fits-all Compilation Optimization across Machine Learning Computers for Inference

Wen, Yuanbo; Guo, Qi; Du, Zidong; Xu, Jian; Zhang, Zhenxing; Hu, Xing; Li, Wei; Zhang, Rui; Wang, Chao; Zhou, Xuehai; Chen, Tianshi

doi:10.1109/tc.2021.3128266

Cited by 2 publications

(2 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…TVM [12] is an end-to-end machine learning compiler framework for CPUs, GPUs, and accelerators. It is an intermediary platform that can integrate various applications and systems, including blockly applications [13], runtime support for Android NNAPI [14], compiler optimizations across many machine learning computers [15] and underlying integration with a GPU [16], [17]. Fig.…”

Section: Background a Tvm And Hybrid Scriptmentioning

confidence: 99%

Case Study: Optimization Methods With TVM Hybrid-OP on RISC-V Packed SIMD

Yu,

Yuan,

Chen

et al. 2024

IEEE Access

View full text Add to dashboard Cite

In recent years, considerable research has focused on the use of custom hardware to accelerate deep learning on edge devices. However, the end-to-end flow of deep learning includes preprocessing and postprocessing. Deep learning hardware accelerators cannot accelerate these operations, which consequently becomes a performance bottleneck in the execution flow. In this study, we propose optimization methods to improve preprocessing and postprocessing at the edge devices. For this purpose, we adopt Tensor Virtual Machine (TVM), an end-to-end machine learning compiler framework. TVM provides hybrid script, which is a frontend language that allows users to write programs for preprocessing and postprocessing. We propose rewriting strategies to improve the performance of operators written in hybrid script through the RISC-V Packed SIMD extension (P extension). RISC-V is an open instruction set architecture (ISA) that provides base instructions and many extensions for different use cases. The P extension defines specific subword single-instruction multiple-data (SIMD) instructions that allow complex computations to be efficiently performed on edge devices. In this study, we design custom instructions based on the RISC-V P extension for rewriting strategies to accelerate deep learning operations. Experimental results indicate that our methods improve performance by a factor of 1.28 to 15.29.

show abstract

Section: Background a Tvm And Hybrid Scriptmentioning

confidence: 99%

Case Study: Optimization Methods With TVM Hybrid-OP on RISC-V Packed SIMD

Yu,

Yuan,

Chen

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…These languages allow programmers to easily specify the essential structure of a problem without concern for low-level details. Crucially, this separation of concerns enables domain-specific compilers [54], [16] to efficiently map programs down to a wide range of idiosyncratic accelerators.…”

Section: Introductionmentioning

confidence: 99%

mlirSynth: Automatic, Retargetable Program Raising in Multi-Level IR Using Program Synthesis

Brauckmann,

Polgreen,

Grosser

et al. 2023

2023 32nd International Conference on Parallel Architectures and Compilation Techniques (PACT)

View full text Add to dashboard Cite

MLIR is an emerging compiler infrastructure for modern hardware, but existing programs cannot take advantage of MLIR's high-performance compilation if they are described in lower-level general purpose languages. Consequently, to avoid programs needing to be rewritten manually, this has led to efforts to automatically raise lower-level to higher-level dialects in MLIR. However, current methods rely on manually-defined raising rules, which limit their applicability and make them challenging to maintain as MLIR dialects evolve.We present mlirSynth -a novel approach which translates programs from lower-level MLIR dialects to high-level ones without manually defined rules. Instead, it uses available dialect definitions to construct a program space and searches it effectively using type constraints and equivalences. We demonstrate its effectiveness by raising C programs to two distinct high-level MLIR dialects, which enables us to use existing high-level dialect specific compilation flows. On Polybench, we show a greater coverage than previous approaches, resulting in geomean speedups of 2.5x (Intel) and 3.4x (AMD) over state-of-the-art compilation flows. mlirSynth also enables retargetability to domain-specific accelerators, resulting in a geomean speedup of 21.6x on a TPU.

show abstract

Enabling One-size-fits-all Compilation Optimization across Machine Learning Computers for Inference

Cited by 2 publications

References 29 publications

Case Study: Optimization Methods With TVM Hybrid-OP on RISC-V Packed SIMD

Case Study: Optimization Methods With TVM Hybrid-OP on RISC-V Packed SIMD

mlirSynth: Automatic, Retargetable Program Raising in Multi-Level IR Using Program Synthesis

Contact Info

Product

Resources

About