Mapping abilities of three-layer neural networks

Arai,

doi:10.1109/ijcnn.1989.118598

Cited by 32 publications

(20 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…(8). typically wishes to match the model complexity with the sample complexity (measured by how much data we have 3 Stating the Problem for Radial Basis on hand) and this problem is well studied [ Broadly speaking, simple models would have high As mentioned before the problem of learning from examapproximation errors but small estimation errors while pies reduces to estimating some target function from a complex models would have low approximation errors set X to a set Y. In most practical cases, such as charbut high estimation errors.…”

Section: Bounding the Generalization Error 2 Another Source Of Errormentioning

confidence: 99%

See 1 more Smart Citation

On the Relationship between Generalization Error, Hypothesis Complexity, and Sample Complexity for Radial Basis Functions

1996

View full text Add to dashboard Cite

Feedforward networks together with their training algorithms are a class of regression techniques that can be used to learn to perform some task from a set of examples. The question of generalization of network performance from a finite training set to unseen data is clearly of crucial importance. In this article we first show that the generalization error can be decomposed into two terms: the approximation error, due to the insufficient representational capacity of a finite sized network, and the estimation error, due to insufficient information about the target function because of the finite number of samples. We then consider the problem of learning functions belonging to certain Sobolev spaces with gaussian radial basis functions. Using the above-mentioned decomposition we bound the generalization error in terms of the number of basis functions and number of examples. While the bound that we derive is specific for radial basis functions, a number of observations deriving from it apply to any approximation technique. Our result also sheds light on ways to choose an appropriate network architecture for a particular problem and the kinds of problems that can be effectively solved with finite resources, i.e., with a finite number of parameters and finite amounts of data.

show abstract

Section: Bounding the Generalization Error 2 Another Source Of Errormentioning

confidence: 99%

“…Drawing upon where x and y range over the generic elements of X and results in approximation theory [551 several researchers 1'. In most cases X will be a subset of a k-dimensional [18, 41,6,44,15,3,57,56,46,76] …”

Section: Random Variables and Probabilitymentioning

confidence: 99%

On the Relationship between Generalization Error, Hypothesis Complexity, and Sample Complexity for Radial Basis Functions

1996

View full text Add to dashboard Cite

show abstract

“…For any given hidden layer function , let denote the basin of attraction of defined by (9) Let also denote the minimum Hamming distance between the prototype and any other prototype (10) where denotes the Hamming distance between and . We note that if , then…”

Section: Design Preliminariesmentioning

confidence: 99%

An efficient learning algorithm for associative memories

Batalama

2000

IEEE Trans. Neural Netw.

View full text Add to dashboard Cite

Associative memories (AMs) can be implemented using networks with or without feedback. In this paper we utilize a two-layer feedforward neural network and propose a new learning algorithm that efficiently implements the association rule of a bipolar AM. The hidden layer of the network employs p neurons where p is the number of prototype patterns. In the first layer, the input pattern activates at most one hidden layer neuron or "winner." In the second layer, the "winner" associates the input pattern to the corresponding prototype pattern. The underlying association principle is minimum Hamming distance and the proposed scheme can be viewed also as an approximately minimum Hamming distance decoder. Theoretical analysis supported by simulations indicates that, in comparison with other suboptimum minimum Hamming distance association schemes, the proposed structure exhibits the following favorable characteristics: 1) It operates in one-shot which implies no convergence-time requirements; 2) it does not require any feedback; and 3) our case studies show that it exhibits superior performance than the popular linear system in a saturated mode (LSSM). The network also exhibits 4) exponential capacity and 5) easy performance assessment (no asymptotic analysis is necessary). Finally, since it does not require any hidden layer interconnections or tree-search operations, it exhibits low structural as well as operational complexity.

show abstract

“…Considerable research has been done in the field of multilayer neural networks with respect to feedforward network abilities (Akaho & Amari, 1990;Arai, 1989;Huang & Huang, 1991;Mehrotra, Mohan, & Ranka, 1991;Heskes, Slijpen, & Kappen, 1992). The dynamic behavior of a sin#e-layer neural network has been well analyzed (Sontag & Sussmann, 1991;Minsky & Papert, 1988).…”

Section: Introductionmentioning

confidence: 99%

Learning behavior and temporary minima of two-layer neural networks

1994

View full text Add to dashboard Cite

This paper presents a mathematical analysis of the occurrence of temporary minima during training of a single-output, two-layer neural network, with learning according to the back-propagation algorithm. A new vector decomposition method is introduced, which simplifies the mathematical analysis of learning of neural networks considerably. The analysis shows that temporary minima are inherent to multilayer networks learning. A number of numerical results illustrate the analytical conclusions.

show abstract

Mapping abilities of three-layer neural networks

Cited by 32 publications

References 1 publication

On the Relationship between Generalization Error, Hypothesis Complexity, and Sample Complexity for Radial Basis Functions

On the Relationship between Generalization Error, Hypothesis Complexity, and Sample Complexity for Radial Basis Functions

An efficient learning algorithm for associative memories

Learning behavior and temporary minima of two-layer neural networks

Contact Info

Product

Resources

About