Locally connected spiking neural networks for unsupervised feature learning

Abstract

In recent years, Spiking Neural Networks (SNNs) have demonstrated great successes in completing various Machine Learning tasks. We introduce a method for learning image features by locally connected layers in SNNs using spike-timingdependent plasticity (STDP) rule. In our approach, sub-networks compete via competitive inhibitory interactions to learn features from different locations of the input space. These Locally-Connected SNNs (LC-SNNs) manifest key topological features of the spatial interaction of biological neurons. We explore biologically inspired ngram classification approach allowing parallel processing over various patches of the the image space. We report the classification accuracy of simple two-layer LC-SNNs on two image datasets, which match the state-of-art performance and are the first results to date. LC-SNNs have the advantage of fast convergence to a dataset representation, and they require fewer learnable parameters than other SNN approaches with unsupervised learning. Robustness tests demonstrate that LC-SNNs exhibit graceful degradation of performance despite the random deletion of large amounts of synapses and neurons.

BioLCNet: Reward-modulated Locally Connected Spiking Neural Networks

“…Very recently, (Pogodin et al 2021) proposed bio-inspired dynamic weight sharing and adding lateral connections to locally-connected layers to achieve the same regularization goals of weight sharing and normal convolutional filters. The first work to integrate a locally-connected (LC) layer into an SNN (Saunders et al 2019) used a network with no hidden layers where the rate-coded input is passed to the output layer via local connections. They exploited recurrent inhibitory connections similar to the ones employed by (Diehl and Cook 2015) to simulate a winner-take-all (WTA) inhibition mechanism in their output.…”

“…One is allowing the weights to have negative values, which corresponds to having inhibitory neurons, as done in the convolutional layers used by (Lee et al 2018). The other is to use a combination of recurrent inhibitory connections and adaptive thresholds as done by (Diehl and Cook 2015;Saunders et al 2018Saunders et al , 2019. In this work, we used the latter approach for our feature extraction LC layer.…”

“…In this work, we clipped the weights to stay in the range [0, 1]. We also employed the normalization technique used by (Saunders et al 2019) and normalized the pre-synaptic weights of each neuron in the LC layer to have a constant mean of c norm at the end of each time step.…”

A heuristic approach to the hyperparameters in training spiking neural networks using spike-timing-dependent plasticity

“…Lee et al [17] described a proposition to develop spike-based backpropagation idea when [26] shown a training solution using equilibrium propagation. Again, Saunders et al [30] described another unsupervised technique for feature training. Spiking networks can be shallow and as deep as their predecessors.…”

A learning-based short-term wind speed forecasting approach through spiking neural networks

“…They are deeply dependent on the biological background of neural works. Among the diversity of neural networks, spiking neural networks (SNNs) are advocated because the kind of networks is more closely mimic natural neural networks, that is, the structure of SNNs is designed to describe realistic brain-like information processing (Bohte et al, 2002;Shrestha and Song, 2017;Saunders et al, 2019). The SNNs-based modelling becomes popular and well-reputed.…”