ChewBaccaNN: A Flexible 223 TOPS/W BNN Accelerator

Abstract

Binary Neural Networks enable smart IoT devices, as they significantly reduce the required memory footprint and computational complexity while retaining a high network performance and flexibility. This paper presents ChewBaccaNN, a 0.7 mm 2 sized binary convolutional neural network (CNN) accelerator designed in GlobalFoundries 22 nm technology. By exploiting efficient data re-use, data buffering, latch-based memories, and voltage scaling, a throughput of 241 GOPS is achieved while consuming just 1.1 mW at 0.4V/154MHz during inference of binary CNNs with up to 7×7 kernels, leading to a peak core energy efficiency of 223 TOPS/W. ChewBaccaNN's flexibility allows to run a much wider range of binary CNNs than other accelerators, drastically improving the accuracy-energy tradeoff beyond what can be captured by the TOPS/W metric. In fact, it can perform CIFAR-10 inference at 86.8% accuracy with merely 1.3 µJ, thus exceeding the accuracy while at the same time lowering the energy cost by 2.8× compared to even the most efficient and much larger analog processing-in-memory devices, while keeping the flexibility of running larger CNNs for higher accuracy when needed. It also runs a binary ResNet-18 trained on the 1000-class ILSVRC dataset and improves the energy efficiency by 4.4× over accelerators of similar flexibility. Furthermore, it can perform inference on a binarized ResNet-18 trained with 8bases Group-Net to achieve a 67.5% Top-1 accuracy with only 3.0 mJ/frame-at an accuracy drop of merely 1.8% from the fullprecision ResNet-18.

Low- and Mixed-Precision Inference Accelerators

“…It is clear that parallelism and data reuse (either in the form of locally buffering or by broadcasting) are the keys to amortizing the memory access cost, which is so much larger than the low-precision arithmetic cost. Techniques to mitigate these costs are to replace SRAM with low-voltage SCM, hard-wire network parameters to enable broadcasting, and use the sliding window principle (like the FMM banks in combination with the crossbar in ChewBaccaNN [1]). In essence, all these solutions boil down to designing the architecture around the data movements in a less-flexible manner.…”

“…A full system-on-a-chip (SoC), implemented in 22nm technology is presented including the accelerator, RISC host processor, and peripherals. • ChewBaccaNN [1] is an architecture for binary neural network inference that exploits efficient data reuse by co-designing the memory hierarchy with the neural network ran on the architecture. The hard-wired kernel size allows efficient data reuse.…”

“…ChewBaccaNN [1] is like XNE an accelerator utilizing binary weights and binary activations. Contrary to XNE, this architecture does not implement a full SoC and is therefore purely based on the accelerator core.…”

How to train accurate BNNs for embedded systems?

“…Replacing a real-valued MAC by XNOR-PopCount is very good for energy efficiency. E.g., in [2] a BNN accelerator is presented that achieves down to merely 4.48 fJ/Op in GF 22nm at 0.4V, where Op is a binary operation (xnor or popcount). The most common network architecture used in BNN papers is illustrated in Fig.…”

“…Similar to ABCNet [26], a weighted sum is used to get to the original feature size. Compared to the per-layer ensemble this ensemble expands the network by a factor 𝑁 rather than 𝑁 2 .…”