Global optimization for artificial neural networks: A tabu search application

Sexton, Randall S.; Alidaee, Bahram; Dorsey, Robert E.; Johnson, John D.

doi:10.1016/s0377-2217(97)00292-0

Cited by 129 publications

(52 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In [140], an improvised TS, called reactive tabu search was used for optimizing weights. Several studies show that TS when used for optimizing FNN weights, outperformed BP and SA algorithm [141,142]. However, SA and TS are single solution based algorithms, which has a limited scope of exploring search space to obtain a global optimal solution.…”

Section: Weight Optimizationmentioning

confidence: 99%

Metaheuristic design of feedforward neural networks: A review of two decades of research

Ojha

Abraham

Snel

2017

Engineering Applications of Artificial Intelligence

414

154

View full text Add to dashboard Cite

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era.

show abstract

Section: Weight Optimizationmentioning

confidence: 99%

Metaheuristic design of feedforward neural networks: A review of two decades of research

Ojha

Abraham

Snel

2017

Engineering Applications of Artificial Intelligence

414

154

View full text Add to dashboard Cite

show abstract

“…This issue is well-known for ANN researchers: consult e.g. Bianchini and Gori (1996), Sexton et al (1998), Abraham (2004), Hamm et al (2007). To overcome this difficulty, the various optimization strategies used in practice to train ANNs include experimental design, sampling by low-discrepancy sequences, theoretically exact global or local scope search approaches, as well as popular heuristics such as evolutionary optimization methods, particle swarm optimization, simulated annealing, tabu search and tunneling functions.…”

Section: Postulating and Calibrating A Model Instancesmentioning

confidence: 99%

“…ANN model parameterization frameworks and numerical studies are presented and discussed e.g. by Watkin (1993), Prechelt (1994), Bianchini and Gori (1996), Sexton et al (1998), Jordanov and Brown (1999), Toh (1999), Ye and Lin (2003), Abraham (2004), Hamm et al (2007).…”

Section: Postulating and Calibrating A Model Instancesmentioning

confidence: 99%

Calibrating artificial neural networks by global optimization

Pintér

2012

Expert Systems with Applications

View full text Add to dashboard Cite

An artificial neural network (ANN) is a computational model − implemented as a computer program − that is aimed at emulating the key features and operations of biological neural networks. ANNs are extensively used to model unknown or unspecified functional relationships between the input and output of a "black box" system. In order to apply such a generic procedure to actual decision problems, a key requirement is ANN training to minimize the discrepancy between modeled and measured system output. In this work, we consider ANN training as a (potentially) multi-modal optimization problem. To address this issue, we introduce a global optimization (GO) framework and corresponding GO software. The practical viability of the GO based approach is illustrated by finding close numerical approximations of (one-dimensional, but non-trivial) functions.

show abstract

“…Engoziner et al [11] presented that BP use some variation of the gradient technique, which is essentially a local optimizing method and thus has some inevitable drawbacks, such as easily trapping into the local optimal and dissatisfying generalization capability. Sexton et al [12] proposed the fact that the gradient descent algorithm may perform poorly even on simple problems when predicting the holdout data. And document [13] suggested that, in interest of mitigating the above limitation, weighted values and thresholds of neurons in BP are optimized by global search algorithms.…”

Section: Introductionmentioning

confidence: 99%

Weights Optimization of Neural Network via Improved Bco Approach

Zhang¹,

Wu²

2008

PIER

View full text Add to dashboard Cite

Abstract-Feed forward neural Network (FNN) has been widely applied to many fields because of its ability to closely approximate unknown function to any degree of desired accuracy. Back Propagation (BP) is the most general learning algorithms, but is subject to local optimal convergence and poor performance even on simple problems when forecasting out of samples. Thus, we proposed an improved Bacterial Chemotaxis Optimization (BCO) approach as a possible alternative to the problematic BP algorithm, along with a novel adaptive search strategy to improve the efficiency of the traditional BCO. Taking the classical XOR problem and sinc function approximation as examples, comparisons were implemented. The results demonstrate that our algorithm is obviously superior in convergence rate and precision compared with other training algorithms, such as Genetic Algorithm (GA) and Taboo Search (TS).

show abstract

Global optimization for artificial neural networks: A tabu search application

Cited by 129 publications

References 26 publications

Metaheuristic design of feedforward neural networks: A review of two decades of research

Metaheuristic design of feedforward neural networks: A review of two decades of research

Calibrating artificial neural networks by global optimization

Weights Optimization of Neural Network via Improved Bco Approach

Contact Info

Product

Resources

About