Performance Comparison of Different Multilayer Perceptron Network Activation Functions in Automated Weather Classification

Abstract

Multilayer perceptron network (MLP) has been recognized as a powerful tool for many applications including classification. Selection of the activation functions in the multilayer perceptron (MLP) network plays an essential role on the network performance. This paper presents a comparison study of two commonly used MLP activation function; sigmoid and hyperbolic tangent for weather classification. Meteorological data such as solar radiation, ambient temperature, current, surface temperature, voltage, wind direction and wind speed data are acquired from a photovoltaic (PV) system. Then, the meteorological data are input to the MLP network to classify the weather condition. In this study, weather conditions are classified into four types; rain, cloudy, dry day and storm. Levenberg-Marquardt algorithm is used to train the MLP network since it is the fastest training and ensure the best converges towards a minimum error. Experimental results show that hyperbolic tangent activation function is more efficient compared to sigmoid activation function. The MLP network using hyperbolic tangent function has achieved higher classification accuracy with less number of hidden nodes compared to sigmoid activation function.

A comparison of artificial neural networks and random forests to predict native fish species richness in Mediterranean rivers

“…The transfer function in the hidden layer was a hyperbolic tangent and an identity function in the case of the output layer. The hyperbolic tangent gave optimal results in previous studies (Isa et al, 2010) in which a performance comparison was carried out to select the best MLP activation function (Olaya-Marín et al, 2012). The Levenberg-Marquardt (LM) optimization algorithm was used to train the candidate models because this algorithm is the fastest method to train neural networks of moderate size (Karul et al, 2000).…”

Modelling critical factors affecting the distribution of the vulnerable endemic Eastern Iberian barbel (Luciobarbus guiraonis) in Mediterranean rivers

“…Several MLP models were built and tested, in order to establish (by systematic trial and error) the optimal number of neurons in the hidden layer and the optimal transfer function in the hidden and output layers. Commonly, transfer functions are nonlinear; they transform the weighted sum of inputs into an output signal (Zhang et al, 1998;Isa et al, 2010) and it is typical to use the same transfer function in hidden and output layers (Lek et al, 2005;Goethals et al, 2007). MLP results are very sensitive to the implemented transfer functions in their layers (Piekniewski & Rybicki, 2004;Isa et al, 2010).…”

“…Commonly, transfer functions are nonlinear; they transform the weighted sum of inputs into an output signal (Zhang et al, 1998;Isa et al, 2010) and it is typical to use the same transfer function in hidden and output layers (Lek et al, 2005;Goethals et al, 2007). MLP results are very sensitive to the implemented transfer functions in their layers (Piekniewski & Rybicki, 2004;Isa et al, 2010). Generally, the selection of a transfer function is based on the best performance by trial and error (Isa et al, 2010), by comparing different transfer functions in the hidden and output layers.…”

“…MLP results are very sensitive to the implemented transfer functions in their layers (Piekniewski & Rybicki, 2004;Isa et al, 2010). Generally, the selection of a transfer function is based on the best performance by trial and error (Isa et al, 2010), by comparing different transfer functions in the hidden and output layers. In this work, two transfer function combinations (hidden layer/output layer) were tested: hyperbolic tangent/linear, and logistic/linear; the combination offering the best performance was selected (Isa et al, 2010;Olaya-Marín et al, 2012).…”

Multi-Objective Optimization by Means of Multi-Dimensional MLP Neural Networks

“…Researchers in various studies have corroborated that the polynomial and spline activation functions have the potential to estimate the highly non-linear systems [20][21][22][23]. Many other works imply that selection of activation functions is dependent on the application [24,25]. Consequently and as a new point of view, if we choose the arbitrary function f (x) as the activation function in the last layer, so that f (x) would meet the just mentioned conditions, then as the energy of the network approaches its minimum, the output of f (x) will converge to its desired objective.…”