Representations and rates of approximation of real-valued Boolean functions by neural networks

Kůrková, Véra; Savický, Petr; Hlavackova, Katerina

doi:10.1016/s0893-6080(98)00039-2

Cited by 52 publications

(55 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hence, any linear combination of n elements of F d belongs to span dn+1Hd . As for any orthonormal basis of a separable Hilbert space, G-variation is equal to l 1 -norm with respect to G [16], [18], we have f F d = f 1,F d , and the statement follows from Proposition 4.1(i) and Corollary 5.2(iii).…”

Section: Convergence Of Minimizing Sequences Formed By Variable-basismentioning

confidence: 92%

“…Equicontinuity follows from the mean value theorem [6, p. 79] and the Cauchy-Schwarz inequality, which together imply that for all f ∈ C, all x ∈ (0, 1)For the proof of Theorem 5.1(i) and (ii) see [13] and [14], respectively; for the proof of Theorem 5.1(iii) see [16,Theorem 3] and [18,Theorem 2.7].…”

Section: Convergence Of Minimizing Sequences Formed By Variable-basismentioning

confidence: 99%

“…; so every function of the Fourier basis F d can be expressed as a linear combination of at most d + 1 signum perceptrons [18]. Hence, any linear combination of n elements of F d belongs to span dn+1Hd .…”

Section: Convergence Of Minimizing Sequences Formed By Variable-basismentioning

confidence: 99%

“…G-variation is a norm on the subspace {f ∈ X : f G < ∞} ⊆ X; for its properties see [15], [17], and [18]. In [16] and [18] it was shown that when G is an orthonormal basis of a separable Hilbert space, G-variation is equal to the l 1 -norm with respect to G, defined for f ∈ X as f 1,G = g∈G |f · g|.…”

Section: Rates Of Decrease Of Infima With Increasing Complexity Of Admentioning

confidence: 99%

“…In [16] and [18] it was shown that when G is an orthonormal basis of a separable Hilbert space, G-variation is equal to the l 1 -norm with respect to G, defined for f ∈ X as f 1,G = g∈G |f · g|. For t > 0, we define…”

Section: Rates Of Decrease Of Infima With Increasing Complexity Of Admentioning

confidence: 99%

See 4 more Smart Citations

Minimization of Error Functionals over Variable-Basis Functions

Kainen¹,

Kurková²,

Sanguineti³

2004

SIAM J. Optim.

View full text Add to dashboard Cite

Abstract. Generalized Tikhonov well-posedness is investigated for the problem of minimization of error functionals over admissible sets formed by variable-basis functions, i.e., linear combinations of a fixed number of elements chosen from a given basis without a prespecified ordering. For variablebasis functions of increasing complexity, rates of decrease of infima of error functionals are estimated. Upper bounds are derived on such rates which do not exhibit the curse of dimensionality with respect to the number of variables of admissible functions. Consequences are considered for Boolean functions and decision trees. 1. Introduction. Functionals defined as distances from (target) sets are called error functionals. Minimization of such functionals occurs in optimization tasks arising in various areas such as decision processes, system identification, machine learning, and pattern recognition.In various applications, admissible solutions over which error functionals are minimized are functions depending on a large number of variables: for example, when routing strategies have to be devised for large-scale communication and transportation networks, when an optimal closed-loop control law has to be determined for a dynamical system with high-dimensional output measurement vector and a large number of decision stages, etc. In the last decades, complex optimization problems of this kind have been approximately solved by searching suboptimal solutions over ad- Neural networks can be studied in a more general context of variable-basis functions, which also include other nonlinear families of functions such as free-node splines and trigonometric polynomials with free frequencies [17]. Families of variable-basis functions are formed by linear combinations of a fixed number of elements chosen from a given basis without a prespecified ordering [16], [17].When admissible functions depend on a large number of variables, implementation of some procedures of approximate optimization may be infeasible due to the "curse of dimensionality" [3]. For example, when optimization is performed over linear combinations of fixed-basis functions, the number of basis functions required to

show abstract

Section: Convergence Of Minimizing Sequences Formed By Variable-basismentioning

confidence: 92%

Section: Convergence Of Minimizing Sequences Formed By Variable-basismentioning

confidence: 99%

Section: Convergence Of Minimizing Sequences Formed By Variable-basismentioning

confidence: 99%

Section: Rates Of Decrease Of Infima With Increasing Complexity Of Admentioning

confidence: 99%

Section: Rates Of Decrease Of Infima With Increasing Complexity Of Admentioning

confidence: 99%

See 3 more Smart Citations

Minimization of Error Functionals over Variable-Basis Functions

Kainen¹,

Kurková²,

Sanguineti³

2004

SIAM J. Optim.

View full text Add to dashboard Cite

show abstract

Bounds on the complexity of neural‐network models and comparison with linear methods

Hlaváčková‐Schindler¹,

Sanguineti

2003

Adaptive Control & Signal

View full text Add to dashboard Cite

A class of non-linear models having the structure of combinations of simple, parametrized basis functions is investigated; this class includes widespread neural networks in which the basis functions correspond to the computational units of a type of networks. Bounds on the complexity of such models are derived in terms of the number of adjustable parameters necessary for a given modelling accuracy. These bounds guarantee a more advantageous tradeoff than linear methods between modelling accuracy and model complexity: the number of parameters may increase much more slowly, in some cases only polynomially, with the dimensionality of the input space in modelling tasks. Polynomial bounds on complexity allow one to cope with the so-called 'curse of dimensionality', which often makes linear methods either inaccurate or computationally unfeasible. The presented results let one gain a deeper theoretical insight into the effectiveness of neural-network architectures, noticed in complex modelling applications.

show abstract

Minimization of empirical error over perceptron networks

Kůrková

Adaptive and Natural Computing Algorithms

View full text Add to dashboard Cite

Representations and rates of approximation of real-valued Boolean functions by neural networks

Cited by 52 publications

References 15 publications

Minimization of Error Functionals over Variable-Basis Functions

Minimization of Error Functionals over Variable-Basis Functions

Bounds on the complexity of neural‐network models and comparison with linear methods

Minimization of empirical error over perceptron networks

Contact Info

Product

Resources

About