A Synaptic Pruning-Based Spiking Neural Network for Hand-Written Digits Classification

Abstract

A spiking neural network model inspired by synaptic pruning is developed and trained to extract features of hand-written digits. The network is composed of three spiking neural layers and one output neuron whose firing rate is used for classification. The model detects and collects the geometric features of the images from the Modified National Institute of Standards and Technology database (MNIST). In this work, a novel learning rule is developed to train the network to detect features of different digit classes. For this purpose, randomly initialized synaptic weights between the first and second layers are updated using average firing rates of pre- and postsynaptic neurons. Then, using a neuroscience-inspired mechanism named, “synaptic pruning” and its predefined threshold values, some of the synapses are deleted. Hence, these sparse matrices named, “information channels” are constructed so that they show highly specific patterns for each digit class as connection matrices between the first and second layers. The “information channels” are used in the test phase to assign a digit class to each test image. In addition, the role of feed-back inhibition as well as the connectivity rates of the second and third neural layers are studied. Similar to the abilities of the humans to learn from small training trials, the developed spiking neural network needs a very small dataset for training, compared to the conventional deep learning methods that have shown a very good performance on the MNIST dataset. This work introduces a new class of brain-inspired spiking neural networks to extract the features of complex data images.

Unsupervised learning in hexagonal boron nitride memristor-based spiking neural networks

“…Synapses connecting neurons with high spiking correlation are preserved, while synapses with poor or uncorrelated spiking activity are pruned [64]. Weight dropout also mitigates overfitting in neural networks trained with large size data sets by preventing unwanted specialization towards details and noise in the training data and allowing better generalization [65].…”

Sparse-Firing Regularization Methods for Spiking Neural Networks with Time-to-First-Spike Coding

“…Moreover, unlike the methods in [26,27], the SSR methods can calculate the gradient without observing the membrane potential, which may simplify the learning system on hardware. Finally, in addition to the reduction in the firing rate, the combination of binarized weights [48] and pruned weights [49,27,50] is expected to make the SNN model more suitable for hardware implementation.…”

A hardware-implemented neural network for the analysis of handwritten numerals

“…Every neuron processes 785 inputs, each input being multiplied by a corresponding weight. The aggregated sum of these products yields a final score, which is subsequently normalized using the sigmoid function [8].…”