Approximation of continuous functions on Rd by linear combinations of shifted rotations of a sigmoid function with and without scaling

Ito, Yoshifusa

doi:10.1016/s0893-6080(05)80009-7

Cited by 119 publications

(44 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…When the activation function is non-linear, for instance to use a binary step function or a sigmoid function instead, the SLFN with at most k hidden neurons can learn k distinct observations with zero error [30]. In such case, the both weights w and β need to be adjusted carefully with various methods to ensure its approximation capability [32].…”

Section: Feedforward Neural Networkmentioning

confidence: 99%

Air quality forecasting using neural networks

Zhao

Heeswijk

Karhunen

2016

2016 IEEE Symposium Series on Computational Intelligence (SSCI)

View full text Add to dashboard Cite

In this thesis project, a special type of neural network: Extreme Learning Machine (ELM) is implemented to predict the air quality based on the air quality time series itself and the external meteorological records. A regularized version of ELM with linear components is chosen to be the main model for prediction. To take full advantage of this model, its hyper-parameters are studied and optimized. Then a set of variables is selected (or constructed) to maximize the performance of ELM, where two different variable selection methods (i.e. wrapper and filtering methods) are evaluated. The wrapper method ELM-based forward selection is chosen for the variable selection. Meanwhile, a feature extraction method (Principal Component Analysis) is implemented in the hope of reducing the candidate meteorological variables for feature selection, which proves to be helpful. At last, with all the parameters being properly optimized, ELM is used for the prediction and generates satisfying results.

show abstract

Section: Feedforward Neural Networkmentioning

confidence: 99%

Air quality forecasting using neural networks

Zhao

Heeswijk

Karhunen

2016

2016 IEEE Symposium Series on Computational Intelligence (SSCI)

View full text Add to dashboard Cite

show abstract

“…In [57], the C -density property was proved for any continuous bounded and nonconstant computational unit, and the L p -density property was proved for any bounded and nonconstant computational unit. The C-density property for functions on the whole R d was investigated in [50] and [51].…”

Section: Being Sigmoidal Is Not Substantialmentioning

confidence: 99%

“…In the papers [51] and [61], various density properties were proved for monotone sigmoidal functions, using only weights with a norm equal to 1. The case of continuous sigmoidal computational units and of weights and thresholds taking only integer values was addressed in [73].…”

Section: Restricting the Parameter Setmentioning

confidence: 99%

Universal Approximation by Ridge Computational Models and Neural Networks: A Survey

Sanguineti

2008

TOAMJ

View full text Add to dashboard Cite

show abstract

“…In many applications, it is convenient to take the activation function σ as a sigmoidal function which is defined as lim t→−∞ σ(t) = 0 and lim t→+∞ σ(t) = 1. The literature on neural networks abounds with the use of such functions and their superpositions (see, e.g., [2,4,6,8,10,11,13,15,20,22,29]). The possibility of approximating a continuous function on a compact subset of the real line or d-dimensional space by SLFNs with a sigmoidal activation function has been well studied in a number of papers.…”

Section: Introductionmentioning

confidence: 99%

“…For example, Stinchcombe and White [34] showed that SLFNs with a polygonal, polynomial spline or analytic activation function and a bounded set of weights have the universal approximation property. Ito [20] investigated this property of networks using monotone sigmoidal functions (tending to 0 at minus infinity and 1 at infinity), with only weights located on the unit sphere. In [16,17,19], one of the coauthors considered SLFNs with weights varying on a restricted set of directions and gave several necessary and sufficient conditions for good approximation by such networks.…”

Section: Introductionmentioning

confidence: 99%

On the approximation by single hidden layer feedforward neural networks with fixed weights

2018

View full text Add to dashboard Cite

Abstract. Feedforward neural networks have wide applicability in various disciplines of science due to their universal approximation property. Some authors have shown that single hidden layer feedforward neural networks (SLFNs) with fixed weights still possess the universal approximation property provided that approximated functions are univariate. But this phenomenon does not lay any restrictions on the number of neurons in the hidden layer. The more this number, the more the probability of the considered network to give precise results. In this note, we constructively prove that SLFNs with the fixed weight 1 and two neurons in the hidden layer can approximate any continuous function on a compact subset of the real line. The applicability of this result is demonstrated in various numerical examples. Finally, we show that SLFNs with fixed weights cannot approximate all continuous multivariate functions.

show abstract

Approximation of continuous functions on Rd by linear combinations of shifted rotations of a sigmoid function with and without scaling

Cited by 119 publications

References 5 publications

Air quality forecasting using neural networks

Air quality forecasting using neural networks

Universal Approximation by Ridge Computational Models and Neural Networks: A Survey

On the approximation by single hidden layer feedforward neural networks with fixed weights

Contact Info

Product

Resources

About