Effective AER Object Classification Using Segmented Probability-Maximization Learning in Spiking Neural Networks

Abstract

Address event representation (AER) cameras have recently attracted more attention due to the advantages of high temporal resolution and low power consumption, compared with traditional frame-based cameras. Since AER cameras record the visual input as asynchronous discrete events, they are inherently suitable to coordinate with the spiking neural network (SNN), which is biologically plausible and energy-efficient on neuromorphic hardware. However, using SNN to perform the AER object classification is still challenging, due to the lack of effective learning algorithms for this new representation. To tackle this issue, we propose an AER object classification model using a novel segmented probability-maximization (SPA) learning algorithm. Technically, 1) the SPA learning algorithm iteratively maximizes the probability of the classes that samples belong to, in order to improve the reliability of neuron responses and effectiveness of learning; 2) a peak detection (PD) mechanism is introduced in SPA to locate informative time points segment by segment, based on which information within the whole event stream can be fully utilized by the learning. Extensive experimental results show that, compared to state-of-the-art methods, not only our model is more effective, but also it requires less information to reach a certain level of accuracy.

Event-based Action Recognition Using Motion Information and Spiking Neural Networks

“…This event-based representation is inherently suitable to cooperate with the spiking neural network (SNN) since SNN also has the event-based property [Hu et al, 2018]. SNN uses discrete spikes to transmit information between units which mimics the behavior of biological neural systems.…”

“…Existing works of SNNs cooperating with event-based cameras mainly focus on the object recognition tasks [Orchard et al, 2015;Xiao et al, 2019;Liu et al, 2020b]. However, since the event-based camera naturally captures movements in the visual scene, it is a good fit for the action recognition task.…”

“…Humans can recognize actions accurately, which motivates us to explore the biological visual cortex to gain experience for event-based recognition. The visual cortex is organized in two different pathways [Jhuang et al, 2007]. One is a ventral stream dealing with shape information, which has been widely used in the existing spiking object recognition models [Orchard et al, 2015;Liu et al, 2020b].…”

“…The visual cortex is organized in two different pathways [Jhuang et al, 2007]. One is a ventral stream dealing with shape information, which has been widely used in the existing spiking object recognition models [Orchard et al, 2015;Liu et al, 2020b]. The other is a dorsal stream involved with the analysis of motion information.…”

“…Specifically, we first adopt motion-sensitive neurons to estimate optical flow for the purpose of local motion (direction and speed) perception. Then, we perform a motion pooling and a spatial pooling to mitigate the effect of the aperture problem [Orchard et al, 2013] and increase the spatial invariance respectively. As the final stage of the architecture, a SNN classifier in which the spiking neurons are fully connected to the previous pooling layer, is adopted as global perception for recognition results.…”