Evolutionary Multi-objective Optimization of Spiking Neural Networks

Jin, Yaochu; Wen, Ruojing; Sendhoff, Bernhard

doi:10.1007/978-3-540-74690-4_38

Cited by 18 publications

(12 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It has been shown that the multiobjective approach is more promising in alleviating forgetting than its single-objective counterpart. The idea of Pareto-optimality can also be extended to the study on connectivity and complexity [91], [92] of general networks, and to the research on structure and functionality of spiking neural networks [93], [94]. His current research interests include topics from systems biology and computational intelligence such as evolutionary system design and structure optimization of adaptive systems.…”

Section: Discussionmentioning

confidence: 99%

Pareto-Based Multiobjective Machine Learning: An Overview and Case Studies

Jin

Sendhoff

2008

IEEE Trans. Syst., Man, Cybern. C

322

View full text Add to dashboard Cite

Abstract-Machine learning is inherently a multiobjective task. Traditionally, however, either only one of the objectives is adopted as the cost function or multiple objectives are aggregated to a scalar cost function. This can be mainly attributed to the fact that most conventional learning algorithms can only deal with a scalar cost function. Over the last decade, efforts on solving machine learning problems using the Pareto-based multiobjective optimization methodology have gained increasing impetus, particularly due to the great success of multiobjective optimization using evolutionary algorithms and other population-based stochastic search methods. It has been shown that Pareto-based multiobjective learning approaches are more powerful compared to learning algorithms with a scalar cost function in addressing various topics of machine learning, such as clustering, feature selection, improvement of generalization ability, knowledge extraction, and ensemble generation. One common benefit of the different multiobjective learning approaches is that a deeper insight into the learning problem can be gained by analyzing the Pareto front composed of multiple Pareto-optimal solutions. This paper provides an overview of the existing research on multiobjective machine learning, focusing on supervised learning. In addition, a number of case studies are provided to illustrate the major benefits of the Pareto-based approach to machine learning, e.g., how to identify interpretable models and models that can generalize on unseen data from the obtained Pareto-optimal solutions. Three approaches to Pareto-based multiobjective ensemble generation are compared and discussed in detail. Finally, potentially interesting topics in multiobjective machine learning are suggested.

show abstract

Section: Discussionmentioning

confidence: 99%

Pareto-Based Multiobjective Machine Learning: An Overview and Case Studies

Jin

Sendhoff

2008

IEEE Trans. Syst., Man, Cybern. C

322

View full text Add to dashboard Cite

show abstract

“…The experimental results have proved that the Memetic Harmony Search Multi-Objective Differential Evolution with Evolving Spiking Neural Network (MEHSMODE-ESNN) gives better results in terms of accuracy and network structure. Unlike previous study mentioned earlier in Jin [14] and other single objective studies [18,19], this paper deals with an improved multi objective method to obtain simple and accurate ESNN. The proposed method evolves toward optimal values defined by several objectives with model accuracy and ESNN's structure to improve performance for classification problems.…”

Section: Introductionmentioning

confidence: 99%

“…In spite of the fact that one Multi-Objective Evolutionary Algorithms (MOEAs) was used and only for SpikeProp learning, most of the related work which use Multi-Objective Genetic Algorithms (MOGAs) is by Jin et al [14]. Both classification performance and connectivity of SNN with latency coding are optimized by MOGA.…”

Section: Introductionmentioning

confidence: 99%

Memetic Harmony Search Algorithm Based on Multi-objective Differ-ential Evolution of Evolving Spiking Neural Networks

Saleh¹,

Shamsuddin²

2015

Int J Swarm Intel Evol Comput

View full text Add to dashboard Cite

Spiking neural network (SNN) plays an essential role in classification problems. Although there are many models of SNN, Evolving Spiking Neural Network (ESNN) is widely used in many recent research works. Evolutionary algorithms, mainly differential evolution (DE) have been used for enhancing ESNN algorithm. However, many realworld optimization problems include several contradictory objectives. Rather than single optimization, Multi-Objective Optimization (MOO) can be utilized as a set of optimal solutions to solve these problems. In this paper, Harmony Search (HS) and memetic approach was used to improve the performance of MOO with ESNN. Consequently, Memetic Harmony Search Multi-Objective Differential Evolution with Evolving Spiking Neural Network (MEHSMODE-ESNN) was applied to improve ESNN structure and accuracy rates. Standard data sets from the UCI machine learning are used for evaluating the performance of this enhanced multi objective hybrid model. The experimental results have proved that the Memetic Harmony Search Multi-Objective Differential Evolution with Evolving Spiking Neural Network (MEHSMODE-ESNN) gives better results in terms of accuracy and network structure. Unlike previous study mentioned earlier in Jin [14] and other single objective studies [18,19], this paper deals with an improved multi objective method to obtain simple and accurate ESNN. The proposed method evolves toward optimal values defined by several objectives with model accuracy and ESNN's structure to improve performance for classification problems. The remaining parts of this paper are organized as follows: Methods including: Evolving Spiking Neural Network, Multi Objective Differential Evolution are presented in section 2. In addition, section 3 clarifies the proposed method Memetic Harmony Search Multi-Objective Differential Evolution with Evolving Spiking Neural Network (MEHSMODE-ESNN) used in this paper, Experimental design is discussed in section 4, section 5 explains in detail the results and discussion, and finally, section 6 concludes the paper with future works. MethodsThis section reviews the vital foundation of evolving spiking neural network (ESNN) and discusses the evolutionary algorithms that have been utilized for enhancement. In the first part, an introduction of ESNN, neuron coding, learning method and ESNN design, and its algorithm is presented. The second part focuses on the algorithms which are used for improvement of ESNN. The concepts and methods of multi objective optimization (MOO) are highlighted. After that, the literature focuses on the working of differential evolution, harmony search (HS) algorithm and memetic approach which is used to improve and enhance the performance of classification . Evolving spiking neural network (ESNN)Currently, several enhancements of SNN have been proposed. Wysoski improved one of these new models known as Evolving Spiking Neural Network (ESNN) [20]. Generally, ESNN used the principles of evolving connectionist systems (ECOS) where neurons are created incrementall...

show abstract

“…However, it was shown in [13] and in our previous work [14], [15] that the rate of convergence may be increased by using techniques such as “behavioral memory” developed by de Garis [16] and “staged evolution” developed by Lewis [17]. GAs have previously been used as learning rules for spiking neural networks acting as classifiers [18], [19]. The work presented in this paper differs in that GAs are used to configure the dynamics of pattern generator networks whose outputs are sustained rhythmic patterns.…”

Section: Introductionmentioning

confidence: 99%

Optimization Methods for Spiking Neurons and Networks

Russell

Orchard

Dong

et al. 2010

IEEE Trans. Neural Netw.

View full text Add to dashboard Cite

Spiking neurons and spiking neural circuits are finding uses in a multitude of tasks such as robotic locomotion control, neuroprosthetics, visual sensory processing, and audition. The desired neural output is achieved through the use of complex neuron models, or by combining multiple simple neurons into a network. In either case, a means for configuring the neuron or neural circuit is required. Manual manipulation of parameters is both time consuming and non-intuitive due to the nonlinear relationship between parameters and the neuron’s output. The complexity rises even further as the neurons are networked and the systems often become mathematically intractable. In large circuits, the desired behavior and timing of action potential trains may be known but the timing of the individual action potentials is unknown and unimportant, whereas in single neuron systems the timing of individual action potentials is critical. In this paper, we automate the process of finding parameters. To configure a single neuron we derive a maximum likelihood method for configuring a neuron model, specifically the Mihalas–Niebur Neuron. Similarly, to configure neural circuits, we show how we use genetic algorithms (GAs) to configure parameters for a network of simple integrate and fire with adaptation neurons. The GA approach is demonstrated both in software simulation and hardware implementation on a reconfigurable custom very large scale integration chip.

show abstract

Evolutionary Multi-objective Optimization of Spiking Neural Networks

Cited by 18 publications

References 20 publications

Pareto-Based Multiobjective Machine Learning: An Overview and Case Studies

Pareto-Based Multiobjective Machine Learning: An Overview and Case Studies

Memetic Harmony Search Algorithm Based on Multi-objective Differ-ential Evolution of Evolving Spiking Neural Networks

Optimization Methods for Spiking Neurons and Networks

Contact Info

Product

Resources

About