C++OpenCL4TVM: Support C++OpenCL Kernel for TVM NN Operators

Chang, Po-Yao; Chen, Tai‐Liang; Huang, Yu-Tse; Yu, Meng-Shiun; Lee, Jenq-Kuen

doi:10.1145/3529538.3530001

Cited by 1 publication

(1 citation statement)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…TVM [12] is an end-to-end machine learning compiler framework for CPUs, GPUs, and accelerators. It is an intermediary platform that can integrate various applications and systems, including blockly applications [13], runtime support for Android NNAPI [14], compiler optimizations across many machine learning computers [15] and underlying integration with a GPU [16], [17]. Fig.…”

Section: Background a Tvm And Hybrid Scriptmentioning

confidence: 99%

Case Study: Optimization Methods With TVM Hybrid-OP on RISC-V Packed SIMD

Yu,

Yuan,

Chen

et al. 2024

IEEE Access

Self Cite

View full text Add to dashboard Cite

In recent years, considerable research has focused on the use of custom hardware to accelerate deep learning on edge devices. However, the end-to-end flow of deep learning includes preprocessing and postprocessing. Deep learning hardware accelerators cannot accelerate these operations, which consequently becomes a performance bottleneck in the execution flow. In this study, we propose optimization methods to improve preprocessing and postprocessing at the edge devices. For this purpose, we adopt Tensor Virtual Machine (TVM), an end-to-end machine learning compiler framework. TVM provides hybrid script, which is a frontend language that allows users to write programs for preprocessing and postprocessing. We propose rewriting strategies to improve the performance of operators written in hybrid script through the RISC-V Packed SIMD extension (P extension). RISC-V is an open instruction set architecture (ISA) that provides base instructions and many extensions for different use cases. The P extension defines specific subword single-instruction multiple-data (SIMD) instructions that allow complex computations to be efficiently performed on edge devices. In this study, we design custom instructions based on the RISC-V P extension for rewriting strategies to accelerate deep learning operations. Experimental results indicate that our methods improve performance by a factor of 1.28 to 15.29.

show abstract