Improving Classification Accuracy of Feedforward Neural Networks for Spiking Neuromorphic Chips

Abstract

Deep Neural Networks (DNN) achieve human level performance in many image analytics tasks but DNNs are mostly deployed to GPU platforms that consume a considerable amount of power. New hardware platforms using lower precision arithmetic achieve drastic reductions in power consumption. More recently, brain-inspired spiking neuromorphic chips have achieved even lower power consumption, on the order of milliwatts, while still offering real-time processing. However, for deploying DNNs to energy efficient neuromorphic chips the incompatibility between continuous neurons and synaptic weights of traditional DNNs, discrete spiking neurons and synapses of neuromorphic chips need to be overcome. Previous work has achieved this by training a network to learn continuous probabilities, before it is deployed to a neuromorphic architecture, such as IBM TrueNorth Neurosynaptic System, by random sampling these probabilities. The main contribution of this paper is a new learning algorithm that learns a TrueNorth configuration ready for deployment. We achieve this by training directly a binary hardware crossbar that accommodates the TrueNorth axon configuration constrains and we propose a different neuron model. Results of our approach trained on electroencephalogram (EEG) data show a significant improvement with previous work (76% vs 86% accuracy) while maintaining state of the art performance on the MNIST handwritten data set.

RANC: Reconfigurable Architecture for Neuromorphic Computing

“…No data normalization is applied. We present our comparison with the results of Yepes et al [25] in Table III, and we find that, across the board, we have comparable performance to published work. We attribute slight deviations in network accuracy to our particular choice of the Adam optimizer and the stochastic nature of neural network training.…”

“…We begin by discussing the Xilinx Alveo-based runtime framework used for conducting each of these experiments. Then, we recreate two TrueNorth-based case studies described by Yepes et al [25] on the MNIST and EEG datasets. As these both involve mapping SNNs to RANC, we utilize RANC's Tensorflow integration as shown in Figure 5 to create networks and map them to both our simulation and emulation environments.…”

“…Starting with the MNIST case study, the architectures used are equivalent to the 9 and 30 core networks presented in Yepes et al [25]. In these experiments, N (n) = N (a) = 256 to enable comparison with TrueNorth-based results.…”

“…In these experiments, N (n) = N (a) = 256 to enable comparison with TrueNorth-based results. As discussed in Yepes et al [25], MNIST images are 28 × 28, and this exceeds the maximum input size of a single core, so they partition the input into windows of either 16×16=256 or 8×8=64 and feed it into multiple cores. The input images are encoded via burst encoding over an input window ranging from 1 to 16 ticks.…”

A Survey on Deep Learning-based Non-Invasive Brain Signals:Recent Advances and New Frontiers

“…(i) Discriminative models. In the beginning, the basic MLP is adopted to classify manually extracted features when deep learning rst arose [234]. Frydenlund et al [51] extracted the average and standard deviation of each EEG band and then fed them into an MLP for emotional a ect estimation.…”