SWPU: A 126.04 TFLOPS/W Edge-Device Sparse DNN Training Processor With Dynamic Sub-Structured Weight Pruning

Wang, Yang; Qin, Yubin; Liu, Leibo; Wei, Shaojun; Yin, Shouyi

doi:10.1109/tcsi.2022.3184175

Cited by 6 publications

(2 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Output sparsity exploitation during the WG stage has big benefits thanks to both useless computation avoidance and memory access removal. For this reason, recent energy-efficient training processors [26,31,51,78] supported triple sparsity exploitation by combining iterative pruning.…”

Section: Pruning-aware Output Zero Skipping During the Wgmentioning

confidence: 99%

“…[29] [26,31,51,78] Sparsity Exploitation [17][18][19] was proposed to unify the data representation method of both input operand and accumulation. Flexpoint [55] tried to substitute FP with FXP representation using a shared exponent management algorithm together for simplification of MAC design, but it failed to reduce the required bit-precision to less than 16-bit.…”

Section: A New Number Representationmentioning

confidence: 99%

See 1 more Smart Citation

Energy-Efficient DNN Training Processors on Micro-AI Systems

Han

Kang

Kim

et al. 2022

IEEE Open J. Solid-State Circuits Soc.

View full text Add to dashboard Cite

Many edge/mobile devices are now able to utilize deep neural networks (DNNs) thanks to the development of mobile DNN accelerators. Mobile DNN accelerators overcame the problems of limited computing resources and battery capacity by realizing energy-efficient inference. However, its passive behavior makes it difficult for DNN to provide active customization for individual users or its service environment. The importance of onchip training is rising more and more to provide active interaction between DNN processors and ever-changing surroundings or conditions. Despite its advantages, the DNN training has more constraints than the inference such that it was considered impractical to be realized on mobile/edge devices. Recently, there are many trials to realize mobile DNN training, and a number of prior works will be summarized. Firstly, it arranges the new challenges of the DNN accelerator induced by training functionality and discusses new hardware features related to the challenges. Secondly, it explains algorithm-hardware cooptimization methods and explains why it becomes mainstream in mobile DNN training research. Thirdly, it compares the main differences between the conventional inference accelerators and recent training processors. Finally, the conclusion is made by proposing the future directions of the DNN training processor in micro-AI systems.

show abstract

Section: Pruning-aware Output Zero Skipping During the Wgmentioning

confidence: 99%