The Evolution of Training Parameters for Spiking Neural Networks with Hebbian Learning

Kozdon, Katarzyna; Bentley, Peter J.

doi:10.1162/isal_a_00055

Cited by 7 publications

(3 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another important feature is spike-timing dependent plasticity (STDP) [34], which can be used to implement Hebbian learning [35], [36]-it is the ability of a synapse to modify its state (typically its weight) depending on local network activity (pre-and postsynaptic spikes). Plasticity is implemented by adding an Update Synapses step to the pipeline.…”

Section: Methodsmentioning

confidence: 99%

Faster and Simpler SNN Simulation with Work Queues

Bautembach,

Oikonomidis,

Kyriazis

et al. 2019

Preprint

View full text Add to dashboard Cite

We present a clock-driven Spiking Neural Network simulator which is up to 3x faster than the state of the art while, at the same time, being more general and requiring less programming effort on both the user's and maintainer's side. This is made possible by designing our pipeline around "work queues" which act as interfaces between stages and greatly reduce implementation complexity. We evaluate our work using three well-established SNN models on a series of benchmarks.

show abstract

Section: Methodsmentioning

confidence: 99%

Faster and Simpler SNN Simulation with Work Queues

Bautembach,

Oikonomidis,

Kyriazis

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…Recent works blending EC and SNNs are mostly focused on evolving a network's weights, using evolutionary approaches as a learning strategy [20,29,30]. Schuman et al [37] proposed Evolutionary Optimization for Neuromorphic Systems, aiming to train spiking neural networks for classification and control tasks, to train under hardware constraints, to evolve a reservoir for a liquid state machine, and to evolve smaller networks using multi-objective optimization.…”

Section: Related Workmentioning

confidence: 99%

SPENSER: Towards a NeuroEvolutionary Approach for Convolutional Spiking Neural Networks

Branquinho

Lourenço

Costa

2023

Proceedings of the Companion Conference on Genetic and Evolutionary Computation

View full text Add to dashboard Cite

Spiking Neural Networks (SNNs) have attracted recent interest due to their energy efficiency and biological plausibility. However, the performance of SNNs still lags behind traditional Artificial Neural Networks (ANNs), as there is no consensus on the best learning algorithm for SNNs. Best-performing SNNs are based on ANN to SNN conversion or learning with spike-based backpropagation through surrogate gradients. The focus of recent research has been on developing and testing different learning strategies, with hand-tailored architectures and parameter tuning. Neuroevolution (NE), has proven successful as a way to automatically design ANNs and tune parameters, but its applications to SNNs are still at an early stage. DENSER is a NE framework for the automatic design and parametrization of ANNs, based on the principles of Genetic Algorithms (GA) and Structured Grammatical Evolution (SGE). In this paper, we propose SPENSER, a NE framework for SNN generation based on DENSER, for image classification on the MNIST and Fashion-MNIST datasets. SPENSER generates competitive performing networks with a test accuracy of 99.42% and 91.65% respectively. CCS CONCEPTS• Computing methodologies → Neural networks; Computer vision; • Theory of computation → Evolutionary algorithms; Grammars and context-free languages.

show abstract

“…An important class of learning rules for SNNs are Hebbian learning algorithms, in particular the classical Hebb rule (Hebb 1949;Oja 1982) and its generalisation spike timing dependent plasticity (STDP) (Caporale and Dan 2008;Kozdon and Bentley 2018;Lobov et al 2020;Białas et al 2020;Long 2011). Unlike backpropagationbased algorithms (Shrestha et al 2018;Fang 2021) Hebbian and STDP algorithms can be implemented on neuromorphic hardware and thus benefit from its advantages.…”

Section: Introductionmentioning

confidence: 99%

Information theoretical properties of a spiking neuron trained with Hebbian and STDP learning rules

Chu

2023

Nat Comput

View full text Add to dashboard Cite

Using formal methods complemented by large-scale simulations we investigate information theoretical properties of spiking neurons trained using Hebbian and STDP learning rules. It is shown that weight space contains meta-stable states, which are points where the average weight change under the learning rule vanishes. These points may capture the random walker transiently. The dwell time in the vicinity of the meta-stable state is either quasi-infinite or very short and depends on the level of noise in the system. Moreover, important information theoretic quantities, such as the amount of information the neuron transmits are determined by the meta-stable state. While the Hebbian learning rule reliably leads to meta-stable states, the STDP rule tends to be unstable in the sense that for most choices of hyper-parameters the weights are not captured by meta-stable states, except for a restricted set of choices. It emerges that stochastic fluctuations play an important role in determining which meta-stable state the neuron takes. To understand this, we model the trajectory of the neuron through weight space as an inhomogeneous Markovian random walk, where the transition probabilities between states are determined by the statistics of the input signal.

show abstract

The Evolution of Training Parameters for Spiking Neural Networks with Hebbian Learning

Cited by 7 publications

References 23 publications

Faster and Simpler SNN Simulation with Work Queues

Faster and Simpler SNN Simulation with Work Queues

SPENSER: Towards a NeuroEvolutionary Approach for Convolutional Spiking Neural Networks

Information theoretical properties of a spiking neuron trained with Hebbian and STDP learning rules

Contact Info

Product

Resources

About