Evolving Neural Networks Using the Hybrid of Ant Colony Optimization and BP Algorithms

Liu, Yanpeng; Wu, Minxian; Qian, Jixin

doi:10.1007/11759966_105

Cited by 34 publications

(24 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some researchers have demonstrated that standalone evolutionary training is faster than gradient descent training (Montana and Davis, 1989), whereas other researchers have demonstrated that there is no any significant difference between the two types of training and the difference really depends on the problem (Socha and Blum, 2007). However, hybrid training with a GA (Alba and Chicano, 2004) or ACO (Liu et al, 2006;Socha and Blum, 2007;Mavrovouniotis and Yang, 2013) usually performs better than standalone metaheuristics or gradient descent algorithms. This is due to the fact that metaheuristics are global optimization algorithms and they are less sensitive on the initial condition of training whereas local optimization algorithms find the local optimum in the neighbourhood of the initial weights given.…”

Section: Training Artificial Neural Networkmentioning

confidence: 99%

“…For example, in many cases the initial weights selected for back-propagation may lead to a very poor local optimum. To address this issue, some hybrid training algorithms use the best values obtained from ACO training as the initial weights for back-propagation training (Liu et al, 2006;Mavrovouniotis and Yang, 2013). The idea is to select a promising neighbourhood by ACO training and then search for the optimum by gradient descent training.…”

Section: Training Artificial Neural Networkmentioning

confidence: 99%

“…The solutions are constructed by sampling several probability density functions which are derived from the solution archive. The results of ACO R showed that a standalone ACO training algorithm is outperformed by gradient descent training, whereas a hybrid ACO R with back-propagation A different ACO training was proposed in (Liu et al, 2006) that uses a framework that is close to the original ACO framework rather than the ACO R framework. The key idea is to apply ACO training to find good initial weights for the back-propagation which is applied when ACO training is terminated.…”

Section: The Ant Colony Optimization Training Algorithmmentioning

confidence: 99%

“…First, the hybridization of ACO with back-propagation (Liu et al, 2006) uses the traditional ACO framework, where a population of ants construct solutions (e.g., a combination of connection weights) and update the pheromone table (Dorigo et al, 1996). Recently, an improved variation has been proposed in (Mavrovouniotis and Yang, 2013), which is based on the pheromone update policy of the MAX -MIN ant system (Stützle and Hoos, 1997).…”

mentioning

confidence: 99%

“…The key idea of the aforementioned hybrid ACO with back-propagation training (Mavrovouniotis and Yang, 2013;Liu et al, 2006) is to run ACO for several iterations to select the initial connection weights, and then perform back-propagation training. This paper extends the work in (Mavrovouniotis and Yang, 2013) and applies backpropagation, as a local search improvement, to each solution constructed at each ACO iteration.…”

mentioning

confidence: 99%

See 4 more Smart Citations

Training neural networks with ant colony optimization algorithms for pattern classification

Mavrovouniotis

Yang

2014

Soft Comput

View full text Add to dashboard Cite

Feed-forward neural networks are commonly used for pattern classification. The classification accuracy of feed-forward neural networks depends on the configuration selected and the training process. Once the architecture of the network is decided, training algorithms, usually gradient descent techniques, are used to determine the connection weights of the feed-forward neural network. However, gradient descent techniques often get trapped in local optima of the search landscape. To address this issue, an ant colony optimization (ACO) algorithm is applied to train feed-forward neural networks for pattern classification in this paper. In addition, the ACO training algorithm is hybridized with gradient descent training. Both standalone and hybrid ACO training algorithms are evaluated on several benchmark pattern classification problems, and compared with other swarm intelligence, evolutionary and traditional training algorithms. The experimental results show the efficiency of the proposed ACO training algorithms for feed-forward neural networks for pattern classification.

show abstract

Section: Training Artificial Neural Networkmentioning

confidence: 99%

Section: Training Artificial Neural Networkmentioning

confidence: 99%

Section: The Ant Colony Optimization Training Algorithmmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Training neural networks with ant colony optimization algorithms for pattern classification

Mavrovouniotis

Yang

2014

Soft Comput

View full text Add to dashboard Cite

show abstract

Improving the prediction accuracy of residue solvent accessibility and real‐value backbone torsion angles of proteins by guided‐learning through a two‐layer neural network

2008

View full text Add to dashboard Cite

This paper attempts to increase the prediction accuracy of residue solvent accessibility and real-value backbone torsion angles of proteins through improved learning. Most methods developed for improving the backpropagation algorithm of artificial neural networks are limited to small neural networks. Here, we introduce a guided-learning method suitable for networks of any size. The method employs a part of the weights for guiding and the other part for training and optimization. We demonstrate this technique by predicting residue solvent accessibility and real-value backbone torsion angles of proteins. In this application, the guiding factor is designed to satisfy the intuitive condition that for most residues, the contribution of a residue to the structural properties of another residue is smaller for greater separation in the protein-sequence distance between the two residues. We show that the guided-learning method makes a 2-4% reduction in ten-fold cross-validated mean absolute errors (MAE) for predicting residue solvent accessibility and backbone torsion angles, regardless of the size of database, the number of hidden layers and the size of input windows. This together with introduction of two-layer neural network with a bipolar activation function leads to a new method that has a MAE of 0.11 for residue solvent accessibility, 36° for ψ, and 22° for ϕ. The method is available as a Real-SPINE 3.0 server in http://sparks.informatics.iupui.edu.

show abstract

Ant-based Neural Topology Search (ANTS) for Optimizing Recurrent Networks

ElSaid

Ororbia

Desell

2020

Applications of Evolutionary Computation

View full text Add to dashboard Cite

Evolving Neural Networks Using the Hybrid of Ant Colony Optimization and BP Algorithms

Cited by 34 publications

References 10 publications

Training neural networks with ant colony optimization algorithms for pattern classification

Training neural networks with ant colony optimization algorithms for pattern classification

Improving the prediction accuracy of residue solvent accessibility and real‐value backbone torsion angles of proteins by guided‐learning through a two‐layer neural network

Ant-based Neural Topology Search (ANTS) for Optimizing Recurrent Networks

Contact Info

Product

Resources

About