Efficiently inaccurate approximation of hyperbolic tangent used as transfer function in artificial neural networks

Simos, T. E.; Tsitouras, Charalampos

doi:10.1007/s00521-021-05787-0

Cited by 8 publications

(8 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[23]. The second is Softsign function proposed by [24], Aranda-Ordaz introduced by Gomes et al which is labeled as Aranda [16]. Fourth to seventh functions are the bimodal activation functions proposed by Singh et al and labeled as Bisig1, Bi-sig2, Bi-tanh1, and Bi-tanh2, respectively.…”

Section: B Activation Functionsmentioning

confidence: 99%

Developing Novel Activation Functions Based Deep Learning LSTM for Classification

2022

View full text Add to dashboard Cite

This study proposes novel Long Short-Term Memory (LSTM)-based classifiers through developing the internal structure of LSTM neural networks using 26 state activation functions as alternatives to the traditional hyperbolic tangent (tanh) activation function. The LSTM networks have high performance in solving the vanishing gradient problem that is observed in recurrent neural networks. Performance investigations were carried out utilizing three distinct deep learning optimization algorithms to evaluate the efficiency of the proposed state activation functions-based LSTM classifiers for two different classification tasks. The simulation results demonstrate that the proposed classifiers that use the Modified Elliott, Softsign, Sech, Gaussian, Bitanh1, Bitanh2 and Wave as state activation functions trump the tanh-based LSTM classifiers in terms of classification accuracy. The proposed classifiers are encouraged to be utilized and tested for other classification tasks.INDEX TERMS LSTM, deep neural network, activation Function, tanh gate.

show abstract

Section: B Activation Functionsmentioning

confidence: 99%

Developing Novel Activation Functions Based Deep Learning LSTM for Classification

2022

View full text Add to dashboard Cite

show abstract

“…The hidden layer is core of neural system, which receives data from input layer in the form of weighted sum and produce the output through an activation function. Activation function is sometime called transfer function [38], due to the property of transferring data from hidden layer to output layer. The selection of activation function has a great impact on the capability and performance of neural networks [39].…”

Section: Artificial Neural Networkmentioning

confidence: 99%

“…Generally an activation function is a nonlinear function [40], [41] or orthogonal polynomials [42]. Hyperbolic tangent function, logistic activation function, binary step function [38], [43], [44], Hermite polynomials [45], Legendre polynomials [46], Chebyshev polynomials [47] etc are some examples of activation functions. These activation functions decide whether send data to output layer or not.…”

Section: Artificial Neural Networkmentioning

confidence: 99%

“…These weights are plotted in Figure in the form of bar graph and given in Equation (41). The solutions obtained for singular boundary value problem given in Equation (38) subjected to boundary conditions in (39) are tabulated in Table (7) together with some existing solutions approximated by the compact finite difference method (CFDM) [29], advanced Adomian decomposition method (AADM) [26], differential transform method (DTM) [16], quartic B-spline collocation method (QB S CM) [62] and improved decomposition method (IDM) [20]. The designed scheme is highly accurate when compared to the advanced Adomian decomposition method (AADM) [26].…”

Section: Numerical Experimentationmentioning

confidence: 99%

“…The designed scheme is highly accurate when compared to the advanced Adomian decomposition method (AADM) [26]. The solutions of HHO-IPA are presented in The solutions obtained for singular boundary value problem given in Equation (38) subjected to boundary conditions in (39) are cross examined by getting the absolute errors in our solution. The absolute errors are calculated by using Equation (25).…”

Section: Numerical Experimentationmentioning

confidence: 99%

See 2 more Smart Citations

A Novel Neuroevolutionary Paradigm for Solving Strongly Nonlinear Singular Boundary Value Problems in Physiology

2022

View full text Add to dashboard Cite

In the present research work, we designed a hybrid stochastic numerical solver to investigate nonlinear singular two-point boundary value problems with Neumann and Robin boundary conditions arising in various physical models. In this method, we hybridized Harris Hawks Optimizer with Interior Point Algorithm named HHO-IPA. We construct artificial neural networks (ANNs) model for the problem, and this model is tuned with the proposed scheme. This scheme overcomes the singular behavior of problems. The accuracy and applicability of the method are illustrated by finding absolute errors in the solution. The outcomes are compared with the results present in the literature to demonstrate the effectiveness and robustness of the scheme by considering four different nonlinear singular boundary value problems. Further, the convergence of the scheme is proved by performing computational complexity analysis. Moreover, the graphical overview of statistical analysis is added to our investigation to elaborate further on the scheme's stability, accuracy, and consistency.

show abstract

Real-time guidance for powered landing of reusable rockets via deep learning

Wang

et al. 2022

Neural Comput & Applic

View full text Add to dashboard Cite

Efficiently inaccurate approximation of hyperbolic tangent used as transfer function in artificial neural networks

Cited by 8 publications

References 15 publications

Developing Novel Activation Functions Based Deep Learning LSTM for Classification

Developing Novel Activation Functions Based Deep Learning LSTM for Classification

A Novel Neuroevolutionary Paradigm for Solving Strongly Nonlinear Singular Boundary Value Problems in Physiology

Real-time guidance for powered landing of reusable rockets via deep learning

Contact Info

Product

Resources

About