Visual comparison of performance for different activation functions in MLP networks

On Training Efficiency and Computational Costs of a Feed Forward Neural Network: A Review

“…In [18], the sigmoid activation function is modified by introducing the square of the argument, enhancing the mapping capabilities of the NN. In [19], two activation functions, one based on integration of the triangular function and one on the difference between two sigmoids (log-exponential), are proposed and compared through a barycentric plotting technique, which projects the mapping capabilities of the network in a hyper dimensional cube. The study has shown that log-exponential function has been slowly accelerated but it was effective in MLP network with backpropagation learning.…”

Suitable MLP Network Activation Functions for Breast Cancer and Thyroid Disease Detection

“…Sometimes different activation functions [4][5][6][7][8][9][10][11][12][13][14] are acquired for different networks so that it resulting in better performances. An activation function or transfer functions for the hidden nodes in MLP are needed to introduce nonlinearity into the network.…”

“…This happens due to improper selection of architecture, initialization of weights or data selection. Another factor that affects the training process is the selection of transfer function [10].…”

“…The study has solved multicluster classification and identification problem of XOR logic function, parity generation, handwritten digit recognition, piecewise linear function approximation and sunspot series prediction. Piekniewski and Tybicki [10] employed different activation functions in MLP networks to determine the visual comparison performance. The study has shown that log-exponential function has been slowly accelerated but it was effective in MLP network with back propagation learning.…”

“…However, the most recommended activation function among neural network researchers for MLP network is sigmoid or hyperbolic tangent [6], [10]. The objective of this study is to measure performances of MLP networks on classifying diagnosis data of thyroid disease and breast cancer using MLP networks and Levenberg-Marquardt (LM) as the training algorithm.…”