Evolutionary artificial neural networks by multi-dimensional particle swarm optimization

Kıranyaz, Serkan; İnce, Türker; Yıldırım, Alper; Gabbouj, Moncef

doi:10.1016/j.neunet.2009.05.013

Cited by 236 publications

(111 citation statements)

References 39 publications

(63 reference statements)

Supporting

Mentioning

109

Contrasting

Unclassified

Order By: Relevance

“…Particularly, we can foresee that those problems where a proper guidance mechanism such as FGBF is needed but cannot be applied due to infeasibility problems (e.g. evolutionary ANN applications by the standalone MD-PSO as in [18] and [23]), SAD MD-PSO would be a promising solution to further improve the performance. For those multi-objective problems (MOPs), the application of SAD PSO can be crucial since the goal is to search for a set of Pareto-optimal solutions and hence, the particles must follow (a set of) best guide(s) that will lead them toward the optimal solutions.…”

Section: Discussionmentioning

confidence: 99%

Stochastic approximation driven particle swarm optimization with simultaneous perturbation – Who will guide the guide?

Kıranyaz

İnce

Gabbouj

2011

Applied Soft Computing

Self Cite

View full text Add to dashboard Cite

Section: Discussionmentioning

confidence: 99%

Stochastic approximation driven particle swarm optimization with simultaneous perturbation – Who will guide the guide?

Kıranyaz

İnce

Gabbouj

2011

Applied Soft Computing

Self Cite

View full text Add to dashboard Cite

“…Similarly, in [210,211], a hybrid Taguchi-genetic algorithm was proposed for optimizing the FNN architecture and weights, where authors used a genetic representation of the weights, but they select structure using constructive method (by adding hidden nodes one-by-one). A multidimensional PSO approach was proposed in [212] for constructing FNN automatically by using an architectural (topological) space. Moreover, the individuals in the swarm population were designed in such a way that it optimized both position (weights) and dimension (architecture) of an individual in each iteration.…”

Section: Architecture Plus Weight Optimizationmentioning

confidence: 99%

Metaheuristic design of feedforward neural networks: A review of two decades of research

Ojha

Abraham

Snel

2017

Engineering Applications of Artificial Intelligence

441

154

View full text Add to dashboard Cite

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era.

show abstract

“…Multi-dimensional PSO (MD PSO) [14] is natural extension of the conventional PSO method that allows the particles to make interdimensional jumps and visits any dimension within a given range, [ ]. Thus, the MD PSO searches for the global best solution among several search spaces with different dimensions.…”

Section: Multi-dimensional Particle Swarm Optimizationmentioning

confidence: 99%

“…Also, the computation time in PSO is usually less than in GAs, because all the particles in PSO tend to converge to the best solution rather quickly [16]. In this paper we adapt the multidimensional extension of the basic PSO algorithm (MD PSO, [14]) to optimize the output vector dimension of the synthesized audio features. This voids the need of fixing the dimension of the solution space (corresponding to the dimension of the synthesized vector) in advance.…”

Section: Introductionmentioning

confidence: 99%

Multi-dimensional evolutionary feature synthesis for content-based image retrieval

Kıranyaz

Pulkkinen

İnce

et al. 2011

2011 18th IEEE International Conference on Image Processing

Self Cite

View full text Add to dashboard Cite

Abstract-Although there is a wide variety of low-level audio features for content-based audio indexing and retrieval, they m ay lack the discrimination power needed for accurate description of the aural content, leading into a poor content-based retrieval performance. Furthermore, manual selection of features among a vast collection may easily lead into sub-optimal solutions. In this paper, we propose an evolutionary feature synthesis technique, which co-exists with a feature selection scheme. The synthesis process seeks for the optimal linear / nonlinear operators and feature weights from a pre-defined search space, so as to synthesize a highly discriminative set of new (artificial) features from the set of selected features. The evolutionary search process in the multi-dimensional solution space is based on multi-dimensional particle swarm optimization (MD PSO) algorithm, along with a fractional global best formation (FGBF) technique. Unlike in many existing feature generation approaches found in the literature, the dimension of the synthesized feature vector is also optimized during the process. The synthesized features by the proposed approach are compared with original audio descriptors in an extensive set of retrieval tasks. The experimental results clearly demonstrate a crucial improvement of up to 15-25% in the retrieval performance. Moreover, the proposed synthesis technique surpasses the performance of the artificial neural networks for retrieving accurate audio content.Index Terms-Content based retrieval, Evolutionary computation, Feature extraction, Particle swarm optimization. I. INTRODUCTIONThe features can be numerical or nominal scalars or vectors describing some specific characteristics of the data, such as, in case of audio signals, tonality or fundamental frequency. The accuracy of the content-based audio retrieval is strongly dependent on the description quality and discrimination capability of the features. Unfortunately, despite the enormous number of different feature extraction methods available in the literature, most of them have significant limitations and drawbacks in describing the audio content, so that the current audio content search techniques cannot really cope up with the human perceptual auditory system. This lack of semantic representation, also known as the "semantic gap", has led researchers to search and acquire several promising ideas for improving the discrimination power of the low-level (primitive) features. Another reason is the so-called curse of dimensionality phenomenon [2], which basically states that in high dimensions the data becomes too sparse for any decent statistical or structural analysis. In the feature selection scheme, the feature vector dimension is lowered by selectively choosing an expressive and compact set of features among a possibly much larger original set. Evolutionary algorithms, such as genetic algorithms (GA) [3] and genetic programming (GP) [4], are 978-1-4673-2821-0/13/$31.00 ©2013 IEEE popular among many other feature selection approaches...

show abstract

Evolutionary artificial neural networks by multi-dimensional particle swarm optimization

Cited by 236 publications

References 39 publications

Stochastic approximation driven particle swarm optimization with simultaneous perturbation – Who will guide the guide?

Stochastic approximation driven particle swarm optimization with simultaneous perturbation – Who will guide the guide?

Metaheuristic design of feedforward neural networks: A review of two decades of research

Multi-dimensional evolutionary feature synthesis for content-based image retrieval

Contact Info

Product

Resources

About