Review and Benchmarking of Precision-Scalable Multiply-Accumulate Unit Architectures for Embedded Neural-Network Processing

Abstract

The current trend for deep learning has come with an enormous computational need for billions of Multiply-Accumulate (MAC) operations per inference. Fortunately, reduced precision has demonstrated large benefits with low impact on accuracy, paving the way towards processing in mobile devices and IoT nodes. To this end, various precision-scalable MAC architectures optimized for neural networks have recently been proposed. Yet, it has been hard to comprehend their differences and make a fair judgment of their relative benefits as they have been implemented with different technologies and performance targets. To overcome this, this work exhaustively reviews the state-of-the-art precision-scalable MAC architectures and unifies them in a new taxonomy. Subsequently, these different topologies are thoroughly benchmarked in a 28 nm commercial CMOS process, across a wide range of performance targets, and with precision ranging from 2 to 8 bits. Circuits are analyzed for each precision as well as jointly in practical use cases, highlighting the impact of architectures and scalability in terms of energy, throughput, area and bandwidth, aiming to understand the key trends to reduce computation costs in neural-network processing.Index Terms-ASIC, deep neural networks, precision-scalable circuits, configurable circuits, MAC, multiply-accumulate units. I. INTRODUCTIONE MBEDDED deep learning has gained a lot of attention nowadays due to its broad application prospects and vast potential market. However, the main challenge to embrace this era of edge intelligence comes from the supply-anddemand gap between the limited energy budget of embedded devices, often battery powered, and the computationallyintensive deep-learning algorithms, requiring billions of Multiply-Accumulate (MAC) operations and data movements.To alleviate this unbalanced relationship, many approaches have been investigated at different levels of abstraction. At algorithmic level, researchers have introduced hardware-

A High-Accuracy Hardware-Efficient Multiply–Accumulate (MAC) Unit Based on Dual-Mode Truncation Error Compensation for CNNs

“…Content may change prior to final publication. (12). For a MAC operation that uses TEC-enabled FWBMs, the formulation in (12) can generate a high-accuracy bias term for each FWBM stage.…”

“…(12). For a MAC operation that uses TEC-enabled FWBMs, the formulation in (12) can generate a high-accuracy bias term for each FWBM stage. In terms of MP, TPmajor, and TPminor, the MAC operation in (2) can be rewritten as (13).…”

“…In terms of MP, TPmajor, and TPminor, the MAC operation in (2) can be rewritten as (13). The proposed TEC scheme (Mode 1) truncates the N TPminor terms in (13) and uses their estimated value (expressed in (12)) as a substitute. In (13), the original (Eacc+L)-bit MAC result in (8) can be revised as (14), where a global bias (i.e., BM1) is introduced in the TEC operation using (12).…”

“…The proposed TEC scheme (Mode 1) truncates the N TPminor terms in (13) and uses their estimated value (expressed in (12)) as a substitute. In (13), the original (Eacc+L)-bit MAC result in (8) can be revised as (14), where a global bias (i.e., BM1) is introduced in the TEC operation using (12). In practice, the BM1 value in (14) requires only fractional precision for the w digit using (15), in which a Fw{.}…”

“…Accordingly, Mode 2 TEC scheme is proposed to provide high-accuracy biasing for positive patterns, while disable biasing for zero patterns in each FWBM. Such a function can be directly offered using (12) Table II can be approximated, as shown in (16), based on the probability distribution for nonzero dj in Table I. Consequently, the original E[TPminor] value in (12) can be approximated using (17), where δj indicates whether dj is a nonzero term (i.e., δj = 1 for nonzero dj), which can be mapped to the exclusive-OR operation of (b1, b0) bits in Table I.…”