CANNA: Neural network acceleration using configurable approximation on GPGPU

Imani, Mohsen; Masich, Max; Peroni, Daniel; Wang, Pushen; Rosing, Tajana

doi:10.1109/aspdac.2018.8297401

Cited by 14 publications

(5 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are many other ways of approximating multiplication that had not been applied to deep CNNs, such as [43], [44], [45] among countless others. While we believe that the studied multiplier designs are the most promising, there are most likely other related opportunities for improving CNNs.…”

Section: Related Workmentioning

confidence: 99%

The Effects of Approximate Multiplication on Convolutional Neural Networks

Kim¹,

García

Kim

et al. 2022

IEEE Trans. Emerg. Topics Comput.

View full text Add to dashboard Cite

This paper analyzes the effects of approximate multiplication when performing inferences on deep convolutional neural networks (CNNs). The approximate multiplication can reduce the cost of the underlying circuits so that CNN inferences can be performed more efficiently in hardware accelerators. The study identifies the critical factors in the convolution, fully-connected, and batch normalization layers that allow more accurate CNN predictions despite the errors from approximate multiplication. The same factors also provide an arithmetic explanation of why bfloat16 multiplication performs well on CNNs. The experiments are performed with recognized network architectures to show that the approximate multipliers can produce predictions that are nearly as accurate as the FP32 references, without additional training. For example, the ResNet and Inception-v4 models with Mitch-w6 multiplication produces Top-5 errors that are within 0.2% compared to the FP32 references. A brief cost comparison of Mitch-w6 against bfloat16 is presented where a MAC operation saves up to 80% of energy compared to the bfloat16 arithmetic. The most far-reaching contribution of this paper is the analytical justification that multiplications can be approximated while additions need to be exact in CNN MAC operations.

show abstract

Section: Related Workmentioning

confidence: 99%

The Effects of Approximate Multiplication on Convolutional Neural Networks

Kim¹,

García

Kim

et al. 2022

IEEE Trans. Emerg. Topics Comput.

View full text Add to dashboard Cite

show abstract

“…There are many other ways of approximating multiplica-tion that had not been applied to deep CNNs, such as [43], [44], [45] among countless others. While we believe that the studied multiplier designs are the most promising, there are most likely other related opportunities for improving CNNs.…”

Section: Related Workmentioning

confidence: 99%

Low-power implementation of Mitchell's approximate logarithmic multiplication for convolutional neural networks

Kim¹,

Barrio²,

Hermida³

et al. 2018

2018 23rd Asia and South Pacific Design Automation Conference (ASP-DAC)

View full text Add to dashboard Cite

This paper analyzes the effects of approximate multiplication when performing inferences on deep convolutional neural networks (CNNs). The approximate multiplication can reduce the cost of underlying circuits so that CNN inferences can be performed more efficiently in hardware accelerators. The study identifies the critical factors in the convolution, fully-connected, and batch normalization layers that allow more accurate CNN predictions despite the errors from approximate multiplication. The same factors also provide an arithmetic explanation of why bfloat16 multiplication performs well on CNNs. The experiments are performed with recognized network architectures to show that the approximate multipliers can produce predictions that are nearly as accurate as the FP32 references, without additional training. For example, the ResNet and Inception-v4 models with Mitch-w6 multiplication produces Top-5 errors that are within 0.2% compared to the FP32 references. A brief cost comparison of Mitch-w6 against bfloat16 is presented, where a MAC operation saves up to 80% of energy compared to the bfloat16 arithmetic. The most far-reaching contribution of this paper is the analytical justification that multiplications can be approximated while additions need to be exact in CNN MAC operations.

show abstract

“…To further enhance the associative mechanism, additional approximations not strictly related to the arithmetic MAC approximation can offer an orthogonal dimension for optimization. Bit obfuscation and operand precision lowering were used to relax the matching rules indeed, further increasing the repetitiveness of certain patterns and the probability the required data get available in the associative memory [17][18][19]. To be noted that the error introduced by approximate matching rules might call for auxiliary error-recovering policies through online hardware calibration and/or custom training procedures.…”

Section: Introductionmentioning

confidence: 99%

AxP: A HW-SW Co-Design Pipeline for Energy-Efficient Approximated ConvNets via Associative Matching

Mocerino

Calimera

2021

Applied Sciences

View full text Add to dashboard Cite

The reduction in energy consumption is key for deep neural networks (DNNs) to ensure usability and reliability, whether they are deployed on low-power end-nodes with limited resources or high-performance platforms that serve large pools of users. Leveraging the over-parametrization shown by many DNN models, convolutional neural networks (ConvNets) in particular, energy efficiency can be improved substantially preserving the model accuracy. The solution proposed in this work exploits the intrinsic redundancy of ConvNets to maximize the reuse of partial arithmetic results during the inference stages. Specifically, the weight-set of a given ConvNet is discretized through a clustering procedure such that the largest possible number of inner multiplications fall into predefined bins; this allows an off-line computation of the most frequent results, which in turn can be stored locally and retrieved when needed during the forward pass. Such a reuse mechanism leads to remarkable energy savings with the aid of a custom processing element (PE) that integrates an associative memory with a standard floating-point unit (FPU). Moreover, the adoption of an approximate associative rule based on a partial bit-match increases the hit rate over the pre-computed results, maximizing the energy reduction even further. Results collected on a set of ConvNets trained for computer vision and speech processing tasks reveal that the proposed associative-based hw-sw co-design achieves up to 77% in energy savings with less than 1% in accuracy loss.

show abstract

CANNA: Neural network acceleration using configurable approximation on GPGPU

Cited by 14 publications

References 24 publications

The Effects of Approximate Multiplication on Convolutional Neural Networks

The Effects of Approximate Multiplication on Convolutional Neural Networks

Low-power implementation of Mitchell's approximate logarithmic multiplication for convolutional neural networks

AxP: A HW-SW Co-Design Pipeline for Energy-Efficient Approximated ConvNets via Associative Matching

Contact Info

Product

Resources

About