Ternary neural networks for resource-efficient AI applications

Abstract

The computation and storage requirements for Deep Neural Networks (DNNs) are usually high. This issue limits their deployability on ubiquitous computing devices such as smart phones, wearables and autonomous drones. In this paper, we propose ternary neural networks (TNNs) in order to make deep learning more resource-efficient. We train these TNNs using a teacher-student approach based on a novel, layer-wise greedy methodology. Thanks to our two-stage training procedure, the teacher network is still able to use state-of-the-art methods such as dropout and batch normalization to increase accuracy and reduce training time. Using only ternary weights and activations, the student ternary network learns to mimic the behavior of its teacher network without using any multiplication. Unlike its {-1,1} binary counterparts, a ternary neural network inherently prunes the smaller weights by setting them to zero during training. This makes them sparser and thus more energy-efficient. We design a purpose-built hardware architecture for TNNs and implement it on FPGA and ASIC. We evaluate TNNs on several benchmark datasets and demonstrate up to 3.1× better energy efficiency with respect to the state of the art while also improving accuracy.

Proactive Threat Detection for Connected Cars Using Recursive Bayesian Estimation

“…The applicability of these is problematic because they could consume resources with extended complexity when implemented in live systems [11]. For instance, compiling large and effective Neural Networks require considerable processing power and other hardware computing resources [41]. They also require a pre-generated dataset of known-good/known-bad paths and car movements where 2/3 of the dataset will be used to train the algorithm with remaining 1/3 used for evaluation purposes; accuracy, recall, F1-measure, precision and False-Positive-Rate (FPP) are usually reported as a mean to measure the quality of the tested classifier.…”

Finn

“…For the MNIST dataset, we achieve an FPS which is over 48/6× over the nearest highest throughput design [1] for our SFC-max/LFC-max designs respectively. While our SFC-max design has lower accuracy than the networks implemented by Alemdar et al [1] our LFC-max design outperforms their nearest accuracy design by over 6/1.9× for throughput and FPS/W respectively.…”

“…For the MNIST dataset, we achieve an FPS which is over 48/6× over the nearest highest throughput design [1] for our SFC-max/LFC-max designs respectively. While our SFC-max design has lower accuracy than the networks implemented by Alemdar et al [1] our LFC-max design outperforms their nearest accuracy design by over 6/1.9× for throughput and FPS/W respectively. For other datasets, our CNV-max design outperforms TrueNorth [6] for FPS by over 17/8× for CIFAR-10 / SVHN datasets respectively, while achieving 9.44× higher throughput than the design by Ovtcharov et al [19], and 2.2× over the fastest results reported by Hegde et al [9].…”

“…We refer the reader to the work by Misra and Saha [16] for a comprehensive survey. We cover a recent and representative set of works here, roughly dividing them into four categories based on their basic architecture: 1) a single processing engine [19,30,4,2], usually in the form of a systolic array, which processes each layer sequentially; 2) a streaming architecture [27,1], consisting of one processing engine per network layer; 3) a vector processor [7] with instructions specific to accelerating the primitives operations of convolutions; and 4) a neurosynaptic processor [6], which implements many digital neurons and their interconnecting weights.…”

“…Their designs achieve up to 1.62× the performance density of hand tuned designs. Alemdar et al [1] implement fully-connected ternary-weight neural networks with streaming and report up to 255K frames per second on the MNIST dataset, but concentrate on the training aspect for those networks.…”