A new learning method for inference accuracy, core occupation, and performance co-optimization on TrueNorth chip

Abstract

IBM TrueNorth chip uses digital spikes to perform neuromorphic computing and achieves ultrahigh execution parallelism and power efficiency. However, in TrueNorth chip, low quantization resolution of the synaptic weights and spikes significantly limits the inference (e.g., classification) accuracy of the deployed neural network model. Existing workaround, i.e., averaging the results over multiple copies instantiated in spatial and temporal domains, rapidly exhausts the hardware resources and slows down the computation. In this work, we propose a novel learning method on TrueNorth platform that constrains the random variance of each computation copy and reduces the number of needed copies. Compared to the existing learning method, our method can achieve up to 68.8% reduction of the required neuro-synaptic cores or 6.5× speedup, with even slightly improved inference accuracy.

Classification accuracy improvement for neuromorphic computing systems with one-level precision synapses

“…Comprehensive experiments and analyses on MLP and CNN using MNIST and CIFAR-10 datasets are conducted. Our experiments on MNIST shows negligible accuracy drop (0.19% in CNN), which is much better than the previous work like [5]. From the aspect of the system implementation, there are extensive research studies on binary neural networks deployed in traditional platforms such as CPUs, GPUs and FPGAs.…”

“…Our previous research study [5] specifies for spiking neural networks, where the probability distribution can only be biased to two poles (0 or 1). In this work, we extend the method to memristor-based neural networks adopted by state-of-the-art research and large-scale applications [21].…”

“…For example in our CIFAR-10 experiments, the quantization points in layer conv1, conv2, conv3 and ip1 are [−0.12, 0, 0.12], [−0.08, 0, 0.08], [−0.02, 0, 0.02] and [−0.008, 0, 0.008], respectively. Moreover, we discharge the reliance on the floating-point layer in [5] and explore a pure one-level precision solution. Comprehensive experiments and analyses on MLP and CNN using MNIST and CIFAR-10 datasets are conducted.…”

“…Extensive studies on learning low-resolution synapses have been performed to improve the accuracy of neuromorphic systems. Wen et al presented a new learning method for IBM TrueNorth platform which biases the learned connection probability to binary states (0/1) to hinder accuracy loss [5]. Neural networks with binary resolution are more suitable for generic platforms [6][7] [8].…”

Deep Artificial Neural Networks and Neuromorphic Chips for Big Data Analysis: Pharmaceutical and Bioinformatics Applications

“…As previously mentioned, there are some physical limitations to the current architecture of computers, such as the memory wall [69,70] and energy wall [71], which denote the high power density and low memory bandwidth [72,73]. There are also economic limitations; the cost of designing a chip and the cost of building a fabrication facility are growing alarmingly [74]. However, these limitations will probably be surpassed using other technologies and architectures, like GPU clusters or networks of Neuromorphic chips.…”