Real-time classification and sensor fusion with a spiking deep belief network

Abstract

Deep Belief Networks (DBNs) have recently shown impressive performance on a broad range of classification problems. Their generative properties allow better understanding of the performance, and provide a simpler solution for sensor fusion tasks. However, because of their inherent need for feedback and parallel update of large numbers of units, DBNs are expensive to implement on serial computers. This paper proposes a method based on the Siegert approximation for Integrate-and-Fire neurons to map an offline-trained DBN onto an efficient event-driven spiking neural network suitable for hardware implementation. The method is demonstrated in simulation and by a real-time implementation of a 3-layer network with 2694 neurons used for visual classification of MNIST handwritten digits with input from a 128 × 128 Dynamic Vision Sensor (DVS) silicon retina, and sensory-fusion using additional input from a 64-channel AER-EAR silicon cochlea. The system is implemented through the open-source software in the jAER project and runs in real-time on a laptop computer. It is demonstrated that the system can recognize digits in the presence of distractions, noise, scaling, translation and rotation, and that the degradation of recognition performance by using an event-based approach is less than 1%. Recognition is achieved in an average of 5.8 ms after the onset of the presentation of a digit. By cue integration from both silicon retina and cochlea outputs we show that the system can be biased to select the correct digit from otherwise ambiguous input.

Understanding a Deep Learning Technique through a Neuromorphic System a Case Study with SpiNNaker Neuromorphic Platform

“…One of them that has been implemented on SpiNNaker is the model using RBM [61], which basically implements a spike-based DBN proposed by [62]. Since RBM is a stochastic model, the most convenient method to approach it in a spiking neuron models is by using firing rate.…”

“…The network that consists of 4 layers were used: one input layer with 784 neurons, two hidden layers with 500 neurons each, and a 10 neuron output layer (see Figure 8(b)). Static images are transformed into spike trains by converting each pixel of an MNIST image into a Poisson spike-train with a rate proportional to its intensity, while all firing rates are scaled such that the total firing rate of the population is constant [62]. The firing rate of neuron populations are depicted in Figure 8(c).…”

Asynchronous network of cellular automaton-based neurons for efficient implementation of Boltzmann machines

“…Their applications include approximation of arbitrary functions and probability distributions as well as modeling of biological neuronal activity. Some studies have modified spiking neural network models to act as Boltzmann machines [16][17][18]. They employ stochastic state transitions or strong noise induction to implement the stochastic units.…”

Pragmatic Big Data and smart manufacturing

“…An interesting initiative to notice is the approach that mimics how a doctor or a sound engineer use their eyes to analyse the results of an ECG or a spectrogram, by transforming sensors data to image data [14]. These approaches are also useful when it is necessary to fusion multiple sensors sources [15][16][17].…”