Recurrent Spiking Neural Network Learning Based on a Competitive Maximization of Neuronal Activity

Abstract

Spiking neural networks (SNNs) are believed to be highly computationally and energy efficient for specific neurochip hardware real-time solutions. However, there is a lack of learning algorithms for complex SNNs with recurrent connections, comparable in efficiency with back-propagation techniques and capable of unsupervised training. Here we suppose that each neuron in a biological neural network tends to maximize its activity in competition with other neurons, and put this principle at the basis of a new SNN learning algorithm. In such a way, a spiking network with the learned feed-forward, reciprocal and intralayer inhibitory connections, is introduced to the MNIST database digit recognition. It has been demonstrated that this SNN can be trained without a teacher, after a short supervised initialization of weights by the same algorithm. Also, it has been shown that neurons are grouped into families of hierarchical structures, corresponding to different digit classes and their associations. This property is expected to be useful to reduce the number of layers in deep neural networks and modeling the formation of various functional structures in a biological nervous system. Comparison of the learning properties of the suggested algorithm, with those of the Sparse Distributed Representation approach shows similarity in coding but also some advantages of the former. The basic principle of the proposed algorithm is believed to be practically applicable to the construction of much more complicated and diverse task solving SNNs. We refer to this new approach as “Family-Engaged Execution and Learning of Induced Neuron Groups,” or FEELING.

Spike‐Timing‐Dependent and Spike‐Shape‐Independent Plasticities with Dopamine‐Like Modulation in Nanocomposite Memristive Synapses

“…Also, the NSs with trainable competition between neurons could be implemented in hardware with excitatory and inhibitory neurons. [47]…”

Neurohybrid Memristive CMOS-Integrated Systems for Biosensors and Neuroprosthetics

“…The non-linear behavior of memristive devices in response to electrical pulses together with their unique scalability are the most important advantages that determine a unique possibility of hardware implementation of SNN (Demin and Nekhaev, 2018;Guo et al, 2019) based on the processes of self-organization in neural network architectures and qualitatively different from traditional neural networks (perceptrons). We believe that implementation of brain-like networks of future generations will be based on the stochastic dynamics of memristors and synchronization of neural oscillators.…”

Locally connected spiking neural networks for unsupervised feature learning

“…A network inspired by the autoencoder of the neural networks literature is trained without labels layer-wise to reconstruct the MNIST and CIFAR-10 datasets, and whose output is trained in a supervised fashion to perform classification [27]. A learning algorithm for SNNs is developed on the supposition that neurons tend to maximize their activity in competition with other neurons in [28]. Three-layer networks of Izhikevich regular spiking neurons are trained with STDP on the binary task of distinguishing the digits 0 and 1 as well as on the full set of MNIST digits [29].…”