Survey of Precision-Scalable Multiply-Accumulate Units for Neural-Network Processing

Abstract

The current trend for deep learning has come with an enormous computational need for billions of Multiply-Accumulate (MAC) operations per inference. Fortunately, reduced precision has demonstrated large benefits with low impact on accuracy, paving the way towards processing in mobile devices and IoT nodes. Precision-scalable MAC architectures optimized for neural networks have recently gained interest thanks to their subword parallel or bit-serial capabilities. Yet, it has been hard to make a fair judgment of their relative benefits as they have been implemented with different technologies and performance targets. In this work, run-time configurable MAC units from ISSCC 2017 and 2018 are implemented and compared objectively under diverse precision scenarios. All circuits are synthesized in a 28 nm commercial CMOS process with precision ranging from 2 to 8 bits. This work analyzes the impact of scalability and compares the different MAC units in terms of energy, throughput and area, aiming to understand the optimal architectures to reduce computation costs in neural-network processing.

Review and Benchmarking of Precision-Scalable Multiply-Accumulate Unit Architectures for Embedded Neural-Network Processing

“…The results of this comparative study have highlighted that 2D D&C ST (BitFusion) [14] and SWP ST (ST) [16] have the highest energy efficiency for symmetric scaling scenarios, while 1D D&C ST and 1D 4-bit serial [18] are best for weight-scaling scenarios. In addition to that, 1D D&C SA (DNPU) [13] and 2D D&C SA exceed with high throughput for all scaling scenarios, but suffer together with 2D D&C ST (BitFusion) from large varying bandwidth requirements.…”

“…In addition to that, 1D D&C SA (DNPU) [13] and 2D D&C SA exceed with high throughput for all scaling scenarios, but suffer together with 2D D&C ST (BitFusion) from large varying bandwidth requirements. Despite the recent trend for 1D [17], [18] and 2D [19], [20] serial designs, these are strongly penalized for both throughput and energy efficiency.…”

“…Figs. [16][17][18][19] show the breakdown of memory bandwidth, energy per operation, and silicon area of the circuits. The left subfigures display symmetric scaling scenarios while the right ones display weight-only scaling.…”

“…. Normalized energy/op for symmetric (a) and weight-only (b) scaling in 1D serial[17] and multibit-serial MACs[18]. Beware of the scale.…”

A High-Performance Multiply-Accumulate Unit by Integrating Additions and Accumulations Into Partial Product Reduction Process

“…The multiply-accumulate (MAC) unit is a fundamental block for digital signal processing (DSP) applications [1]. Especially, in recent years, the development of real-time edge applications has become a design trend [2], [3]. Thus, there is a strong demand for high-speed low-power MAC units.…”