How to Reach Real-Time AI on Consumer Devices? Solutions for Programmable and Custom Architectures

Venieris, Stylianos I.; Panopoulos, Ioannis; Leontiadis, Ilias; Venieris, Iakovos S.

doi:10.1109/asap52443.2021.00022

Cited by 5 publications

(1 citation statement)

References 83 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, NLP has been applied to code comments to identify the algorithm [76]. Algorithmic mismatch has been explored on a number of dimensions in relation to AI accelerators [128], and with relation to different FFT API calls [122].…”

Section: Algorithm Identificationmentioning

confidence: 99%

Bind the gap: compiling real software to hardware FFT accelerators

Woodruff

Armengol-Estapé

Ainsworth

et al. 2022

Proceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and Implementation

View full text Add to dashboard Cite

Specialized hardware accelerators continue to be a source of performance improvement. However, such specialization comes at a programming price. The fundamental issue is that of a mismatch between the diversity of user code and the functionality of fixed hardware, limiting its wider uptake.Here we focus on a particular set of accelerators: those for Fast Fourier Transforms. We present FACC (Fourier ACcelerator Compiler), a novel approach to automatically map legacy code to Fourier Transform accelerators. It automatically generates drop-in replacement adapters using Input-Output (IO)-based program synthesis that bridge the gap between user code and accelerators. We apply FACC to unmodified GitHub C programs of varying complexity and compare against two existing approaches. We target FACC to a high-performance library, FFTW, and two hardware accelerators, the NXP PowerQuad and the Analog Devices FFTA, and demonstrate mean speedups of 9x, 17x and 27x respectively.

show abstract

Section: Algorithm Identificationmentioning

confidence: 99%

Bind the gap: compiling real software to hardware FFT accelerators

Woodruff

Armengol-Estapé

Ainsworth

et al. 2022

Proceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and Implementation

View full text Add to dashboard Cite

show abstract

New Processor Architecture and Its Use in Mobile Application Development

Fojtik

2022

Lecture Notes in Networks and Systems

View full text Add to dashboard Cite

FxP-QNet: A Post-Training Quantizer for the Design of Mixed Low-Precision DNNs With Dynamic Fixed-Point Representation

2022

View full text Add to dashboard Cite

Deep neural networks (DNNs) have demonstrated their effectiveness in a wide range of computer vision tasks, with the state-of-the-art results obtained through complex and deep structures that require intensive computation and memory. In the past, graphic processing units enabled these breakthroughs because of their greater computational speed. Now-a-days, efficient model inference is crucial for consumer applications on resource-constrained platforms. As a result, there is much interest in the research and development of dedicated deep learning (DL) hardware to improve the throughput and energy efficiency of DNNs. Low-precision representation of DNN data-structures through quantization would bring great benefits to specialized DL hardware especially when expensive floating-point operations can be avoided and replaced by more efficient fixed-point operations. However, the rigorous quantization leads to a severe accuracy drop. As such, quantization opens a large hyper-parameter space at bit-precision levels, the exploration of which is a major challenge. In this paper, we propose a novel framework referred to as the Fixed-Point Quantizer of deep neural Networks (FxP-QNet) that flexibly designs a mixed low-precision DNN for integer-arithmetic-only deployment. Specifically, the FxP-QNet gradually adapts the quantization level for each data-structure of each layer based on the trade-off between the network accuracy and the low-precision requirements. Additionally, it employs post-training self-distillation and network prediction error statistics to optimize the quantization of floating-point values into fixed-point numbers. Examining FxP-QNet 1 on state-of-the-art architectures and the benchmark ImageNet dataset, we empirically demonstrate the effectiveness of FxP-QNet in achieving the accuracy-compression trade-off without the need for training. The results show that FxP-QNet-quantized AlexNet, VGG-16, and ResNet-18 reduce the overall memory requirements of their full-precision counterparts by 7.16ˆ, 10.36ˆ, and 6.44ˆwith less than 0.95%, 0.95%, and 1.99% accuracy drop, respectively.

show abstract

How to Reach Real-Time AI on Consumer Devices? Solutions for Programmable and Custom Architectures

Cited by 5 publications

References 83 publications

Bind the gap: compiling real software to hardware FFT accelerators

Bind the gap: compiling real software to hardware FFT accelerators

New Processor Architecture and Its Use in Mobile Application Development

FxP-QNet: A Post-Training Quantizer for the Design of Mixed Low-Precision DNNs With Dynamic Fixed-Point Representation

Contact Info

Product

Resources

About