Deep Learning With Spiking Neurons: Opportunities and Challenges

Pfeiffer, Michael; Pfeil, Thomas

doi:10.3389/fnins.2018.00774

Cited by 543 publications

(382 citation statements)

References 150 publications

Supporting

Mentioning

343

Contrasting

Unclassified

Order By: Relevance

“…With the brain's general intelligence in mind, it seems only natural to construct artificial SNNs to advance ML; see [41] for a review. However, training SNNs can be challenging, and their exact operating principles remain elusive; see [42] for a review. In principle, an understanding of the mechanisms by which the brain learns could be used to develop learning algorithms for SNNs.…”

Section: Implications For Learning Of Spiking Neural Networkmentioning

confidence: 99%

Temporal learning of bottom-up connections via spatially nonspecific top-down inputs

Lee

Kim

Vijayan

2019

Preprint

View full text Add to dashboard Cite

In the brain, high-order and low-order areas are connected via bottom-up connections (from low-order to high-order areas) and top-down connections (from high-order to low-order areas). While bottom-up signals are thought to be critical in generating perception, functions of top-down signals have not been clearly delineated. One popular theory is that top-down inputs modify the activity of specific cell assemblies to modulate responses to bottom-up inputs. However, a different line of studies proposes that not all top-down inputs are specifically delivered. As the leading theories cannot account for nonspecific top-down inputs, we seek potential functions of nonspecific topdown signals using network models in our study. Our simulation results suggest that top-down inputs can regulate low-order area responses by providing temporal information even without spatial specificity. Specifically, the temporal information in nonspecific top-down inputs can weaken the undesired bottom-up connections, contributing to bottom-up connections' learning. Further, we found that cortical rhythms (synchronous oscillatory neural responses) are critical in the proposed learning process of bottom-up connections in our model.

show abstract

Section: Implications For Learning Of Spiking Neural Networkmentioning

confidence: 99%

Temporal learning of bottom-up connections via spatially nonspecific top-down inputs

Lee

Kim

Vijayan

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…Combining deep networks with these spikes would open the door for new opportunities ( [9]), among which are biologically more realistic neuronal networks for studying and describing the information processing in the brain as well as interesting technical approaches for improving the operation of such networks (e.g. low power consumption, fast inference, event-driven information processing, and massive parallelization) ( [10]). [11] presented a type of shallow neuronal network that is based on non-negative generative models ( [12,13]).…”

Section: Introductionmentioning

confidence: 99%

Back-propagation learning in deep Spike-By-Spike networks

Rotermund

Pawelzik

2019

Preprint

View full text Add to dashboard Cite

Neural networks are important building blocks in artificial intelligence applications. These technical systems rely on noiseless continuous signals in stark contrast to the discrete action potentials exchanged among the neurons in real brains. A promising approach towards bridging this gap are the Spike-by-Spike (SbS) networks, which were demonstrated to reliably process information based on few stochastic spikes given suitable synaptic weights are provided. What is missing are algorithms for finding weight sets that would optimize the output performance of acyclically connected SbS networks.Here, a learning rule for hierarchically organized SbS networks is derived. It is inspired by the error backpropagation algorithms used in feed-forward neural networks. The properties of this approach are investigated and its functionality demonstrated by simulations. In particular, a Deep Convolutional SbS network for classifying handwritten digits (MNIST) is presented. The resulting classification performance reaches values achieved by standard deep networks that have a similar structure and also use only simple gradient descent. We expect that this new SbS back-prop learning rule will provide a basis for building spiking neuronal network models for research in neuroscience and technical applications, especially when it will become combined with state-of-the-art optimization techniques and implemented on specialized computational hardware.

show abstract

“…SNNs have successfully been implemented to model complicated neuronal dynamics underlying cognition [20]. Remarkable success of deep learning in machine learning for analysis of data from different sources has motivated computer scientists to view deep SNNs as a more efficient replacement of conventional DNNs [21]. A challenge to apply SNNs in deep neural networks is the discontinuous nature of neural communication in SNNs as generating spikes over time.…”

Section: Introductionmentioning

confidence: 99%

“…A challenge to apply SNNs in deep neural networks is the discontinuous nature of neural communication in SNNs as generating spikes over time. Therefore, direct application of back-propagation algorithm in SNNs as used in conventional neural networks is an important challenge for the future spiking deep learning methods [21]. For a review on the challenges and applications of SNNs in deep neural networks, we refer to [21].…”

Section: Introductionmentioning

confidence: 99%

Toward One-Shot Learning in Neuroscience-Inspired Deep Spiking Neural Networks

Faghihi

Molhem

Moustafa

2019

Preprint

View full text Add to dashboard Cite

Conventional deep neural networks capture essential information processing stages in perception. Deep neural networks often require very large volume of training examples, whereas children can learn concepts such as hand-written digits with few examples. The goal of this project is to develop a deep spiking neural network that can learn from few training trials. Using known neuronal mechanisms, a spiking neural network model is developed and trained to recognize hand-written digits with presenting one to four training examples for each digit taken from the MNIST database. The model detects and learns geometric features of the images from MNIST database. In this work, a novel biological back-propagation based learning rule is developed and used to a train the network to detect basic features of different digits. For this purpose, randomly initialized synaptic weights between the layers are being updated. By using a neuroscience inspired mechanism named 'synaptic pruning' and a predefined threshold, some of the synapses through the training are deleted. Hence, information channels are constructed that are highly specific for each digit as matrix of synaptic connections between two layers of spiking neural networks. These connection matrixes named 'information channels' are used in the test phase to assign a digit class to each test image. As similar to humans' abilities to learn from small training trials, the developed spiking neural network needs a very small dataset for training, compared to conventional deep learning methods checked on MNIST dataset.

show abstract

Deep Learning With Spiking Neurons: Opportunities and Challenges

Cited by 543 publications

References 150 publications

Temporal learning of bottom-up connections via spatially nonspecific top-down inputs

Temporal learning of bottom-up connections via spatially nonspecific top-down inputs

Back-propagation learning in deep Spike-By-Spike networks

Toward One-Shot Learning in Neuroscience-Inspired Deep Spiking Neural Networks

Contact Info

Product

Resources

About