A Survey on the Optimization of Artificial Neural Networks Using Swarm Intelligence Algorithms

Abstract

Artificial Neural Networks (ANNs) are becoming increasingly useful in numerous areas as they have a myriad of applications. Prior to using ANNs, the network structure needs to be determined and the ANN needs to be trained. The network structure is usually chosen based on trial and error. The training, which consists of finding the optimal connection weights and biases of the ANN, is usually done using gradient-descent algorithms. It has been found that swarm intelligence algorithms are favorable for both determining the network structure and for the training of ANNs. This is because they are able to determine the network structure in an intelligent way, and they are better at finding the most optimal connection weights and biases during the training as opposed to conventional algorithms. Recently, a number of swarm intelligence algorithms have been employed for optimizing different types of neural networks. However, there is no comprehensive survey on the swarm intelligence algorithms used for optimizing ANNs. In this paper, we present a review of the different types of ANNs optimized using swarm intelligence algorithms, the way the ANNs are optimized, the different swarm intelligence algorithms used, and the applications of the ANNs optimized by swarm intelligence algorithms.INDEX TERMS Artificial neural network, swarm intelligence, optimization. I. INTRODUCTIONArtificial Neural Networks (ANNs) are computational models that simulate the biological neural network that constitutes the human brain to generate inferences based on certain given information. They are suitable for both supervised and unsupervised learning for solving a myriad of classification, regression, clustering, and association problems in a multitude of areas. Notably, ANN has been a prominent algorithm in the domain of machine learning, and has paved the way for the advancement in multiple areas such as natural language processing, fraud detection, computational biology, computer vision, unassisted control of vehicles, speech recognition, medical diagnosis and recommendation systems [1]. Recently, ANNs have been applied for making decisions in healthcare organizations [2], for forecasting the energy use in buildings [3], for the development of greenhouse technol-The associate editor coordinating the review of this manuscript and approving it for publication was Yeliz Karaca .

Enhanced Dung Beetle Optimization Algorithm for Practical Engineering Optimization

“…where Z b denotes the global optimal position, and Ub b and Lb b denote the upper and lower bounds of the optimal foraging area. The small dung beetle position is updated as shown in Equation (7).…”

“…where r 5 is a random number between (0, 1) and θ ∈ (0, π). Different from the original algorithm, in this strategy, the position update of the small dung beetle in the foraging phase will be determined by the random number generated pseudo-randomly in the interval (0, 1), and when the random number is less than 0.5, the position strategy is carried out according to Equation (11); vice versa, the position update is carried out according to Equation (7). This strategy not only improves global search efficiency, but also helps the algorithm to jump out of the local optimum, which lays the foundation for exact optimality search.…”

“…Population-based intelligent optimization algorithms have the potential for global search capability, allowing them to efficiently find the optimal solution to optimization problems, and thus have become a hotspot for scholarly research today [6][7][8]. Li et al proposed an enhanced snake optimization algorithm (ESO) that has the capability of fast convergence when solving four real engineering optimization problems [9].…”

A modified weighted chimp optimization algorithm for training feed-forward neural network

“…In 2023 KUMAR [76] use ZEALOUS-PSO to train multilayer perceptron neural networks. Emambocus in 2023 made a Survey on training neural network using different types of optimization algorithms [77]. And others researchers have recently been using recent-swarm intelligence algorithms for training feedforward neural networks [78][79][80][81][82][83][84].…”

Enhanced Sea Horse Optimization Algorithm for Hyperparameter Optimization of Agricultural Image Recognition

“…In recent years, the superior performance of swarm intelligence algorithms in the field of optimization has gradually attracted extensive attention, especially in the field of hyperparameter optimization [5]. Bahaa et al used the improved swarm intelligence optimization algorithm to optimize the convolutional neural network with hyperparameters, so as to construct a new model APSO-WOA-CNN [6].…”