7.1 A 3.4-to-13.3TOPS/W 3.6TOPS Dual-Core Deep-Learning Accelerator for Versatile AI Applications in 7nm 5G Smartphone SoC

Lin, Chien-Hung; Cheng, Chih-Chung; Tsai, Yi-Min; Hung, Sheng-Je; Kuo, Yu‐Ting; Wang, Perry H; Tsung, Pei-Kuei; Hsu, Jeng-Yun; Lai, Wei–Chih; Liu, Chia-Hung; Wang, Shaoyu; Kuo, Chin-Hua; Chang, Chih Hsiang; Lee, Ming-Hsien; Lin, Tsung-Yao; Chen, Chih‐Cheng

doi:10.1109/isscc19947.2020.9063111

Cited by 68 publications

(21 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…NeuroVectorizer [4] uses DRL to predict the appropriate parameters for effective vectorization of computation. Reference [5] provides a brief description of using RL for tiling problems. However, the paper is focused on the hardware architecture of the accelerator and the details on the algorithm are not sufficient to present a justified comparison.…”

Section: Reinforcement Learning (Rl) Is the Branch Of Machinementioning

confidence: 99%

An Automated Approach to Accelerate DNNs on Edge Devices

Chugh

Mitra

Deshwal

et al. 2021

2021 IEEE International Symposium on Circuits and Systems (ISCAS)

View full text Add to dashboard Cite

Deployment of Deep Neural Networks (DNNs) on edge devices can significantly increase the utility of DNNs for a variety of applications. However executing DNN models on the edge device is still a major challenge, as heavy computation and memory bandwidth requirements of such models limit their adoption. Employing a highly optimized code for DNN model execution can easily enable many more use-cases than currently possible. However, current strategies are still based on manual optimization for efficient resource utilization. This is not only cumbersome but also requires a high level of expert intervention in the rapidly changing DNN Model landscape. In this work, we provide an automated way of optimizing Convolutional Neural Network (CNN) models using Deep Reinforcement Learning (DRL) algorithm. The experiments with our DRL technique demonstrate 1.85×, 1.58×, 1.64× speedup in execution time for MobileNetV1, MobileNetV2 and Efficientnet-lite0 CNN models respectively on Mobile CPU devices.

show abstract

Section: Reinforcement Learning (Rl) Is the Branch Of Machinementioning

confidence: 99%

An Automated Approach to Accelerate DNNs on Edge Devices

Chugh

Mitra

Deshwal

et al. 2021

2021 IEEE International Symposium on Circuits and Systems (ISCAS)

View full text Add to dashboard Cite

show abstract

“…MediaTekDLA [20] 3.52 2.125 be significantly reduced on stereo depth estimation networks, our pruning alone sets a new state-of-the-art for ResNet on both CIFAR10 and ImageNet. • We show that pruning before quantization not only can increase sparsity, but also accuracy, because pruned weights cannot induce quantization noise at a later stage.…”

Section: Introductionmentioning

confidence: 99%

Joint Pruning & Quantization for Extremely Sparse Neural Networks

Yu,

Wu,

Klopp

et al. 2020

Preprint

View full text Add to dashboard Cite

We investigate pruning and quantization for deep neural networks. Our goal is to achieve extremely high sparsity for quantized networks to enable implementation on low cost and low power accelerator hardware. In a practical scenario, there are particularly many applications for dense prediction tasks, hence we choose stereo depth estimation as target.We propose a two stage pruning and quantization pipeline and introduce a Taylor Score alongside a new finetuning mode to achieve extreme sparsity without sacrificing performance.Our evaluation does not only show that pruning and quantization should be investigated jointly, but also shows that almost 99% of memory demand can be cut while hardware costs can be reduced up to 99.9%. In addition, to compare with other works, we demonstrate that our pruning stage alone beats the state-of-the-art when applied to ResNet on CIFAR10 and ImageNet.

show abstract

“…Recently, many mobile devices have realized 2-D and 3-D vision tasks using deep neural network (DNN) such as photography improvement [10], visual question and answering [26], and AR/VR [28]. A mobile AP in a mobile device [12], [16], [24], [28], which consists of a CPU, a GPU, and application IPs, performs the various functions in a single chip. Then, the mobile AP integrates a neural processing unit (NPU) to accelerate the DNNs in a real-time.…”

Section: Introductionmentioning

confidence: 99%

Energy-efficient Dense DNN Acceleration with Signed Bit-slice Architecture

Im¹,

Park²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

As the number of deep neural networks (DNNs) to be executed on a mobile system-on-chip (SoC) increases, the mobile SoC suffers from the real-time DNN acceleration within its limited hardware resources and power budget. Although the previous mobile neural processing units (NPUs) take advantages of low-bit computing and exploitation of the sparsity, it is incapable of accelerating high-precision and dense DNNs. This paper proposes energy-efficient signed bit-slice architecture which accelerates both high-precision and dense DNNs by exploiting a large number of zero values of signed bit-slices. Proposed signed bit-slice representation (SBR) changes signed 11112 bitslice to 00002 by borrowing a 1 value from its lower order of bit-slice. As a result, it generates a large number of zero bitslices even in dense DNNs. Moreover, it balances the positive and negative values of 2's complement data, allowing bit-slice based output speculation which pre-computes high order of bit-slices and skips the remaining dense low order of bit-slices. The signed bit-slice architecture compresses and skips the zero input signed bit-slices, and its zero skipping unit also supports the output skipping by masking the speculated inputs as zero. Additionally, the heterogeneous network-on-chip (NoC) benefits exploitation of data reusability and reduction of transmission bandwidth. The paper introduces a specialized instruction set architecture (ISA) and a hierarchical instruction decoder for the control of the signed bit-slice architecture. Finally, the signed bit-slice architecture outperforms the previous bit-slice accelerator, Bitfusion, over ×3.65 higher area-efficiency, ×3.88 higher energyefficiency, and ×5.35 higher throughput.

show abstract

7.1 A 3.4-to-13.3TOPS/W 3.6TOPS Dual-Core Deep-Learning Accelerator for Versatile AI Applications in 7nm 5G Smartphone SoC

Cited by 68 publications

References 4 publications

An Automated Approach to Accelerate DNNs on Edge Devices

An Automated Approach to Accelerate DNNs on Edge Devices

Joint Pruning & Quantization for Extremely Sparse Neural Networks

Energy-efficient Dense DNN Acceleration with Signed Bit-slice Architecture

Contact Info

Product

Resources

About