Comparing Evolutionary Strategy Algorithms for Training Spiking Neural Networks

Altamirano, José S.; Ornelas, Manuel; Espinal, Andrés; Santiago, Raul; Soberanes, Héctor José Puga; Carpío, Martín; Tostado, Sergio

doi:10.13053/rcs-96-1-1

Cited by 6 publications

(10 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The work in [8] trains with the ABC algorithm a one-layer spiking neural network to reduce the dimension of complex datasets into 3D spaces in a linear way. In improving the training of spiking systems, similar researchers are found in [9,10] a decade apart. Both use evolutionary algorithms.…”

Section: Introductionmentioning

confidence: 72%

“…On the other hand, training spiking systems based on metaheuristics could be divided into two branches, firing rate and spike-time delay. The works in [6][7][8] belong to the first category meanwhile those cited in [9][10][11] to the second one. The cuckoo search algorithm in [6], which uses a random walk strategy, achieves the spiking training phase on six complex databases.…”

Section: Introductionmentioning

confidence: 99%

“…Both use evolutionary algorithms. The research contribution in [9] comes from its parallel version and that in [10] due to its comparison among using three evolutionary versions. The work in [11] deals with an advanced training technique, where a grammatical evolution engine guided outmost the performance.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Approaching Optimal Nonlinear Dimensionality Reduction by a Spiking Neural Network

et al. 2021

View full text Add to dashboard Cite

This work deals with the presentation of a spiking neural network as a means for efficiently solving the reduction of dimensionality of data in a nonlinear manner. The underneath neural model, which can be integrated as neuromorphic hardware, becomes suitable for intelligent processing in edge computing within Internet of Things systems. In this sense, to achieve a meaningful performance with a low complexity one-layer spiking neural network, the training phase uses the metaheuristic Artificial Bee Colony algorithm with an objective function from the principals in the machine learning science, namely, the modified Stochastic Neighbor Embedding algorithm. To demonstrate this fact, complex benchmark data were used and the results were compared with those generated by a reference network with continuous-sigmoid neurons. The goal of this work is to demonstrate via numerical experiments another method for training spiking neural networks, where the used optimizer comes from metaheuristics. Therefore, the key issue is defining the objective function, which can relate optimally the information at both sides of the spiking neural network. Certainly, machine learning techniques have advanced in defining efficient loss functions that can become suitable objective function candidates in the metaheuristic training phase. The practicality of these ideas is shown in this article. We use MSE values for evaluating the relative quality of the results and also co-ranking matrices.

show abstract

Section: Introductionmentioning

confidence: 72%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Approaching Optimal Nonlinear Dimensionality Reduction by a Spiking Neural Network

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Evolutionary Algorithms such as Genetic Programing and Evolutionary Strategies are employed to define design traits in Artificial Neural Networks such as training [16,17] and topology [18]. Furthermore, utilizing Grammatical Evolution as a mean to define multiple design criteria such as topologies and neural model parameters of second-generation ANNs is considered in [19], and for third-generation ANNs in [20].…”

Section: Related Workmentioning

confidence: 99%

Evolving Random Topologies of Spiking Neural Networks for Pattern Recognition

López-Vázquez¹,

Ornelas-Rodríguez²,

Espinal

et al. 2019

8th International Conference on Soft Computing, Artificial Intelligence and Applications

Self Cite

View full text Add to dashboard Cite

Artificial Neural Networks (ANNs) have been successfully used in Pattern Recognition tasks. Evolutionary Spiking Neural Networks (ESNNs) constitute an approach to design thirdgeneration ANNs (also known as Spiking Neural Networks, SNNs) involving Evolutionary Algorithms (EAs) to govern some intrinsic aspects of the networks, such as topology, connections and/or parameters. Concerning the practicality of the networks, a rather simple standard is commonly used; restricted feed-forward fully-connected network topologies deprived from more complex connections are usually considered. Notwithstanding, a wider prospect of configurations in contrast to standard network topologies is available for research. In this paper, ESNNs are evolved to solve pattern classification tasks, using an EA-based algorithm known as Grammatical Evolution (GE). Experiments demonstrate competitive results and a distinctive variety of network designs when compared to a more traditional approach to design ESNNs.

show abstract

“…In [44–48], the synaptic weights of a single spiking neuron, e.g., integrate and fire model [13] or Izhikevich model [49], are calibrated by means of algorithms such as differential evolution (DE) [50], particle swarm optimization (PSO) [51], cuckoo search algorithm (CSA) [52], or genetic algorithm (GA) [53] to perform classification tasks; the spiking neuron performs the classification by using the firing rate encoding scheme as the similarity criterion in order to assign the class to which an input pattern belongs. Other works, in [43, 54–57], three-layered feedforward SNNs with synaptic connections were implemented, which are formed by a weight and a delay, to solve supervised classification problems through the use of time-to-first-spike as a classification criterion; in these works, the training has been carried out by means of evolutionary strategy (ES) [58, 59] and PSO algorithms. An extension of previous works is made in [60, 61], where the number of hidden layers and their computing units are defined by grammatical evolution (GE) [62] besides the metaheuristic learning.…”

Section: Introductionmentioning

confidence: 99%

Evolutionary Spiking Neural Networks for Solving Supervised Classification Problems

López-Vázquez

Ornelas-Rodríguez

Espinal

et al. 2019

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

This paper presents a grammatical evolution (GE)-based methodology to automatically design third generation artificial neural networks (ANNs), also known as spiking neural networks (SNNs), for solving supervised classification problems. The proposal performs the SNN design by exploring the search space of three-layered feedforward topologies with configured synaptic connections (weights and delays) so that no explicit training is carried out. Besides, the designed SNNs have partial connections between input and hidden layers which may contribute to avoid redundancies and reduce the dimensionality of input feature vectors. The proposal was tested on several well-known benchmark datasets from the UCI repository and statistically compared against a similar design methodology for second generation ANNs and an adapted version of that methodology for SNNs; also, the results of the two methodologies and the proposed one were improved by changing the fitness function in the design process. The proposed methodology shows competitive and consistent results, and the statistical tests support the conclusion that the designs produced by the proposal perform better than those produced by other methodologies.

show abstract

Comparing Evolutionary Strategy Algorithms for Training Spiking Neural Networks

Cited by 6 publications

References 11 publications

Approaching Optimal Nonlinear Dimensionality Reduction by a Spiking Neural Network

Approaching Optimal Nonlinear Dimensionality Reduction by a Spiking Neural Network

Evolving Random Topologies of Spiking Neural Networks for Pattern Recognition

Evolutionary Spiking Neural Networks for Solving Supervised Classification Problems

Contact Info

Product

Resources

About