Extended Mixture of MLP Experts by Hybrid of Conjugate Gradient Method and Modified Cuckoo Search

Abstract

This paper investigates a new method for improving the learning algorithm of Mixture of Experts (ME) model using a hybrid of Modified Cuckoo Search (MCS) and Conjugate Gradient (CG) as a second order optimization technique. The CG technique is combined with Back-Propagation (BP) algorithm to yield a much more efficient learning algorithm for ME structure. In addition, the experts and gating networks in enhanced model are replaced by CG based Multi-Layer Perceptrons (MLPs) to provide faster and more accurate learning. The CG is considerably depends on initial weights of connections of Artificial Neural Network (ANN), so, a metaheuristic algorithm, the so-called Modified Cuckoo Search is applied in order to select the optimal weights. The performance of proposed method is compared with Gradient Decent Based ME (GDME) and Conjugate Gradient Based ME (CGME) in classification and regression problems. The experimental results show that hybrid MSC and CG based ME (MCS-CGME) has faster convergence and better performance in utilized benchmark data sets.

Global Warming: Predicting OPEC Carbon Dioxide Emissions from Petroleum Consumption Using Neural Network and Hybrid Cuckoo Search Algorithm

“…Among the CS parameters used in the experiment trials, the values suggested by Yang and Deb [ 36 ] were found to be the best settings for the CS. Many literature [ 61 – 63 ] adopted the CS settings proposed by Yang and Deb [ 36 ] for the execution of CS because of their performance.…”

A Review of the Development and Applications of the Cuckoo Search Algorithm

“…Clearly, there would be some benefit to applying some of the modifications discussed above which have a positive impact on performance to these applications, e.g., the increased convergence rate of modified cuckoo search (Walton et al, 2011b) or finding a better optimum using hybridization with gradient-based methods (Salimi et al, 2012). In the most part, these tend to be less concerned with the cuckoo analogy and more with making additions to improve performance.…”

“…When applying optimization techniques to training a neural network, Salimi et al (2012) proposed a simple hybridization between the modified cuckoo search (Walton et al, 2011a,b) and a conjugate gradient method. They were training a neural network for classification and pattern recognition purposes.…”

Stochastic Fractal Search: A powerful metaheuristic algorithm

“…Cuckoo Search (CS) [2] is another successful metaheuristic which mimics the cuckoo behavior reproduction strategy. Over the last decades, uses of metaheuristic algorithms have increased [10][11][12][13][14][15][16][17]. These algorithms are used to solve complex computational optimization problems, however, fast convergence along with accuracy is not guaranteed.…”