Optimizing the Learning Process of Feedforward Neural Networks Using Lightning Search Algorithm

Abstract

Evolutionary Neural Networks are proven to be beneficial in solving challenging datasets mainly due to the high local optima avoidance. Stochastic operators in such techniques reduce the probability of stagnation in local solutions and assist them to supersede conventional training algorithms such as Back Propagation (BP) and Levenberg-Marquardt (LM). According to the No-Free-Lunch (NFL), however, there is no optimization technique for solving all optimization problems. This means that a Neural Network trained by a new algorithm has the potential to solve a new set of problems or outperform the current techniques in solving existing problems. This motivates our attempts to investigate the efficiency of the recently proposed Evolutionary Algorithm called Lightning Search Algorithm (LSA) in training Neural Network for the first time in the literature. The LSA-based trainer is benchmarked on 16 popular medical diagnosis problems and compared to BP, LM, and 6 other evolutionary trainers. The quantitative and qualitative results show that the LSA algorithm is able to show not only better local solutions avoidance but also faster convergence speed compared to the other algorithms employed. In addition, the statistical test conducted proves that the LSA-based trainer is significantly superior in comparison with the current algorithms on the majority of datasets.

A Hybrid Lightning Search Algorithm-Simplex Method for Global Optimization

“…And Figures 23-40 are the ANOVA test of the global minimum from 20 independent runs. As can be seen from Figures 5-22, the convergence accuracy of LSA-SM 8 Discrete Dynamics in Nature and Society 18 , LSA-SM and GSA converge to the global optimal value. In addition, LSA-SM converges faster than other algorithms in all convergence graphs.…”

“…LSA was used to improve the artificial neural network (LSA-ANN) for the home energy management scheduling controller for the residential demand response strategy in 2016 [17]. LSA was also used to optimize the learning process of feedforward neural networks [18]. Because the LSA algorithm simulates the fast propagation characteristics of lightning, it has high convergence rate and strong robustness.…”

Evolving neural networks using bird swarm algorithm for data classification and regression applications

“…Recently, many new meta-heuristic algorithms have been used for learning such biogeography-based optimizer (BBO) [64], Moth-flame optimization [91], multi-verse optimizer (MVO) [27], Grey Wolf optimizer (GWO) [63], and many others [8,9,26,28,29,38].…”

Estimating ARMA Model Parameters of an Industrial Process Using Meta-Heuristic Search Algorithms

“…Various Meta-Heuristic algorithms were explored to solve different types of problems such as global function optimization [20], optimizing neural networks [21][22][23][24], software effort estimation [25], and parameter estimation problem for manufacturing processes. Due to space constraints, we focus only on closely related work of based estimation problem for manufacturing processes that used nature-inspired algorithms.…”