Multi-Layer Perceptron Training Optimization Using Nature Inspired Computing

Abstract

Although the multi-layer perceptron (MLP) neural networks provide a lot of flexibility and have proven useful and reliable in a wide range of classification and regression problems, they still have limitations. One of the most common is associated with the optimization algorithm used to train them. The most commonly used training method is stochastic gradient descent with backpropagation (or backpropagation for short) because it is mathematically tractable (given that the activation functions are differentiable). However, backpropagation is not guaranteed to find the globally optimal set of weights and biases. As a result, the MLP is often incapable of obtaining a desirable solution to the problem. Clonal selection algorithms (CSA) are optimization procedures that effectively explore a complex and large space to find values near the global optimum. Consequently, CSA can be used to solve the problem of training MLP networks. This paper presents a novel implementation of CSA for training MLP architectures to solve real-world problems such as breast cancer diagnosis, active sonar target classification, wheat classification, and flower classification. The CSA is used to find the optimal weights and biases that will significantly increase the classification accuracy of the MLP. The performance of our proposed approach is compared with other popular training methods: genetic algorithm (GA), ant colony optimization (ACO), particle swarm optimization (PSO), Harris hawks optimization (HHO), moth-flame optimization (MFO), flower pollination algorithm (FPA), and backpropagation (BP). The comparison is benchmarked using five classification datasets: Iris Flower, Sonar, Wheat Seeds, Breast Cancer Wisconsin, and Haberman's Survival. Comparative study results illustrate the improvements in MLP performance gained by using CSA over other training methods, and hence it can be considered a competitive approach to training MLP networks when solving real-world applications in various disciplines.

Evaluation of Prediction Models of the Microwire EDM Process of Inconel 718 Using ANN and RSM Methods

“…In the second part of building the prediction models for the investigated process, the ANN was used. Currently, there are many different types of ANN [ 50 , 51 , 52 , 53 ]. One type of ANN that allows for the complex nonlinear nature of the process is the multilayer perceptron network (MLP).…”

A supervised diagnostic experiment of resistance variable multifault locations in a mine ventilation system

“…MLP is a supervised learning algorithm that is a feed-forward network and supports multilabel classification problem solving 26 – 28 . Given a resistance variable multifault feature set and its corresponding fault labels, it can learn to obtain a nonlinear function approximator for multifault location diagnosis.…”

Intelligent Financial Forecasting With an Improved Chemical Reaction Optimization Algorithm Based Dendritic Neuron Model

“…Different ML models such as random forest and gradient boosting approach are used by Yoon for real GDP growth forecasting [44]. Methods such as MLP [45,46], SVR [47], ANN [48,49] are proven successful in capturing the nonlinearities coupled with FTS. However, multiple hidden layers in conventional ANNs leads to longer convergence speed, stuck in local optimum due to traditional back propagation-based learning, and offer black-box visualization thus, paves the path towards designing flat and simple ANNs.…”