Using Algorithmic Transformations and Sensitivity Analysis to Unleash Approximations in CNNs at the Edge

Abstract

Previous studies have demonstrated that, up to a certain degree, Convolutional Neural Networks (CNNs) can tolerate arithmetic approximations. Nonetheless, perturbations must be applied judiciously, to constrain their impact on accuracy. This is a challenging task, since the implementation of inexact operators is often decided at design time, when the application and its robustness profile are unknown, posing the risk of over-constraining or over-provisioning the hardware. Bridging this gap, we propose a two-phase strategy. Our framework first optimizes the target CNN model, reducing the bitwidth of weights and activations and enhancing error resiliency, so that inexact operations can be performed as frequently as possible. Then, it selectively assigns CNN layers to exact or inexact hardware based on a sensitivity metric. Our results show that, within a 5% accuracy degradation, our methodology, including a highly inexact multiplier design, can reduce the cost of MAC operations in CNN inference up to 83.6% compared to state-of-the-art optimized exact implementations.

Overflow-free Compute Memories for Edge AI Acceleration

“…On the other side of the coin, quantization can be harnessed as an optimization opportunity, as operations on aggressively quantized values can be supported with a paucity of hardware resources. Quantization is particularly of interest in the AI domain, where computations are robust towards perturbations, and often approximate results (e.g., for object classification tasks) are acceptable [23] [20]. Indeed, highly aggressive homogeneous [13] or heterogeneous [21] quantization schemes have been proposed in the literature.…”