Constructive Feedforward Neural Networks Using Hermite Polynomial Activation Functions

Ma, Liying; Khorasani, K.

doi:10.1109/tnn.2005.851786

Cited by 121 publications

(76 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The activation function is used to calculate the output response of neuron in which the sum of the weighted input signal is applied with an activation to obtain the response. Different activation or transfer functions have been proposed by different scholars and researchers satisfactory results such as Lorentzian transfer functions [128], Max-Piecewise-Linear (MPWL) Neural Network for function approximation [221], non-polynomial activation functions [33], Hermite Polynomial [132], Gaussian bars [85], hybridization of various functions such as polynomial, periodic, sigmoidal and Gaussian functions [159], two new activation functions labelled sincos and sinc [63], Hybridization of used complementary log-log and probit functions [78]. Giraud in [77] proposed a new class of sigmoidal functions which has proven satisfied with universal approximation theorem requirement.…”

Section: Neural Network (Nn)mentioning

confidence: 99%

A Holistic Review of Soft Computing Techniques

Philip¹,

Mom²,

Igwue³

2017

ACM

View full text Add to dashboard Cite

Due to notable technological convergence that brought about exponential growth in computer world, SoftComputing (SC) has played a vital role with automation capability features to new levels of complex applications. In this research paper, the authors reviewed journals related to the subject matter with the aim of striking a convincing balance between a system that is capable of tolerance to uncertainty, imprecision, approximate reasoning and partial truth to achieve tractability, robustness, economy of communication, high machine intelligence quotient (MIQ), low cost solution and better rapport with reality to conventional techniques. This paper gives an insight on four major consortiums of SC that sprang from the concept of cybernetics, explores and reviews the different techniques, methodologies; application areas and algorithms are formulated to give an idea on how these computing techniques are applied to create intelligent agents to solve a variety of problems. The mechanisms highlighted can serve as an inspiration platform and awareness to new and old researchers that are not or fully grounded in this unique area of research and to create avenue in order to fully embrace the techniques in research communities.

show abstract

Section: Neural Network (Nn)mentioning

confidence: 99%

A Holistic Review of Soft Computing Techniques

Philip¹,

Mom²,

Igwue³

2017

ACM

View full text Add to dashboard Cite

show abstract

“…47, authors propose a growing network training strategy based on Hermite polynomial activation functions instead of sigmoid activation functions.…”

Section: 39 40mentioning

confidence: 99%

An Incremental Framework Based on Cross-Validation for Estimating the Architecture of a Multilayer Perceptron

Aran

Yıldız

Alpaydın

2009

Int. J. Patt. Recogn. Artif. Intell.

View full text Add to dashboard Cite

We define the problem of optimizing the architecture of a multilayer perceptron (MLP) as a state space search and propose the MOST (Multiple Operators using Statistical Tests) framework that incrementally modifies the structure and checks for improvement using cross-validation. We consider five variants that implement forward/backward search, using single/multiple operators and searching depth-first/breadth-first. On 44 classification and 30 regression datasets, we exhaustively search for the optimal and evaluate the goodness based on: (1) Order, the accuracy with respect to the optimal and (2) Rank, the computational complexity. We check for the effect of two resampling methods (5 × 2, ten-fold cv), four statistical tests (5 × 2 cv t, ten-fold cv t, Wilcoxon, sign) and two corrections for multiple comparisons (Bonferroni, Holm). We also compare with Dynamic Node Creation (DNC) and Cascade Correlation (CC). Our results show that: (1) On most datasets, networks with few hidden units are optimal, (2) forward searching finds simpler architectures, (3) variants using single node additions (deletions) generally stop early and get stuck in simple (complex) networks, (4) choosing the best of multiple operators finds networks closer to the optimal, (5) MOST variants generally find simpler networks having lower or comparable error rates than DNC and CC.

show abstract

“…But the number of the network nodes may be too high about some simple systems. Besides, there are other growing strategies for constructing the RBF neural network (Liying Ma & K. Khorasani, 2005;R. Sentiono, 2001; S. S. Ge, F. Hong, & T. H. Lee, 2003).…”

Section: Introductionmentioning

confidence: 99%