A Comparison Between Two Approaches to Optimize Weights of Connections in Artificial Neural Networks

Abstract

Artificial neural networks (ANNs) have been used for estimation in numerous areas. Raising the accuracy of ANNs is always one of the important challenges, which is generally defined as a non-linear optimization problem. The aim of this optimization is to find better values for the weights of the connections and biases in ANN because they seriously affect the efficiency. This study uses two approaches to do such optimization in an ANN. For this aim, we create a feed-forward backpropagation ANN using the functions of MATLAB's deep learning toolbox. To improve its accuracy, in the first approach, we use the Levenberg -Marquardt algorithm (LMA) for training, which is available in MATLAB's deep learning toolbox. In the second approach, we optimize the values of weights and biases of ANN with Genetic Algorithm (GA) and Particle Swarm Optimization (PSO), available in MATLAB's global optimization toolbox. Then, we assess the accuracy of estimation for the trained ANNs. In this way, for the first time in the literature, we compare these methods for the optimization of an ANN. The used data sets are also available in MATLAB. Based on the acquired results, in some data sets, training with LMA, and for some others training with PSO cause the best results, however, training with LMA is faster, significantly. Although the used approaches and the obtained conclusions are beneficial for researchers that work in this field, they have some limitations. For instance, since only the functions and data sets from MATLAB are used, it can only serve as an example for researchers.