An Efficient Accelerator for Sparse Convolutional Neural Networks

Multiply-and-Fire (MNF): An Event-driven Sparse Neural Network Accelerator

“…However, these deep networks [14,21,25,33,36] can have multiple hidden layers, millions of parameters, billions of operations and require tremendous storage and intense computation resources, making it difficult to realize energy-efficient and high-performance solutions. To address this issue, several model compression techniques [10,17,18,20,26,38], efficient dataflow techniques [32,35], and dataflow accelerators [1,3,5,7,8,15,16,19,28,31,39,40,41,42,43] have been proposed and widely investigated in recent years. * These authors contributed equally to this work.…”

“…In particular, exploiting high sparsity in CNNs has emerged as a promising approach for achieving energy-efficient and high-performance CNN solutions [1,3,5,7,8,15,16,19,28,31,39,40,41,42,43]. Several sparse CNN (s-CNN) accelerators have recently been proposed to exploit the structured [28,43] as well as the unstructured sparsity [1,3,5,7,8,15,16,19,31,39,40,41,42] in both CNN model parameters and its activations.…”

“…In particular, exploiting high sparsity in CNNs has emerged as a promising approach for achieving energy-efficient and high-performance CNN solutions [1,3,5,7,8,15,16,19,28,31,39,40,41,42,43]. Several sparse CNN (s-CNN) accelerators have recently been proposed to exploit the structured [28,43] as well as the unstructured sparsity [1,3,5,7,8,15,16,19,31,39,40,41,42] in both CNN model parameters and its activations. Although this works [1,3,5,7,8,15,16,19,28,31,39,40,41,42,43] outperform dense CNN accelerators in terms of energy efficiency and performance, they use compressed vector-vector multiplication (VVM) or matrix-vector multiplication (MVM) engines based on traditional dataflow techniques such as weight stationary, input stationary, output stationary, and row stationary techniques…”

“…Several sparse CNN (s-CNN) accelerators have recently been proposed to exploit the structured [28,43] as well as the unstructured sparsity [1,3,5,7,8,15,16,19,31,39,40,41,42] in both CNN model parameters and its activations. Although this works [1,3,5,7,8,15,16,19,28,31,39,40,41,42,43] outperform dense CNN accelerators in terms of energy efficiency and performance, they use compressed vector-vector multiplication (VVM) or matrix-vector multiplication (MVM) engines based on traditional dataflow techniques such as weight stationary, input stationary, output stationary, and row stationary techniques [32,35]. When the sparsity of inputs or weights becomes excessively large (≥ 60%-90%), previous VVM and MVM accelerators suffer from low utilization issues.…”

RSNN: A Software/Hardware Co-Optimized Framework for Sparse Convolutional Neural Networks on FPGAs

“…A preliminary conference version of our work was reported in [23]. We enhanced that work with a new loadbalance aware pruning method for better overall sparse computation efficiency as well as a kernel transformation method for arbitrary CONV kernels here.…”