Approximation of Sobolev classes by polynomials and ridge functions

Abstract

Let W r p (B d ) be the usual Sobolev class of functions on the unit ball B d in R d , and W •,r p (B d ) be the subclass of all radial functions in W r p (B d ). We show that for the classes W •,r p (B d ) and W r p (B d ), the orders of best approximation by polynomials in L q (B d ) coincide. We also obtain exact orders of best approximation in L 2 (B d ) of the classes W •,r p (B d ) by ridge functions and, as an immediate consequence, we obtain the same orders in L 2 (B d ) for the usual Sobolev classes W r p (B d ).

Realizing Data Features by Deep Nets

“…Smoothness or regularity is a widely used feature that has been adopted in a vast literature [3], [4], [15], [16], [28], [29], [49]. To present the approximation result, we at first introduce the following definition.…”

“…Mathematically, rotationinvariant property corresponds to a radial function which is by definition a function whose value at each point depends only on the distance between that point and the origin. In the nice papers [15], [16], shallow nets were proved to be incapable of embodying rotation-invariance features. To show the power of depth in approximating radial functions, we present the definition of smooth radial function as follows.…”

Deep neural networks for rotation-invariance approximation and learning

“…We point out that the above Lipschitz continuous assumption is standard for radial basis functions (RBF's) in Approximation Theory, and was adopted in [13,14] to quantify the approximation abilities of polynomials and ridge functions. For U, V ⊆ L p (B d ) and 1 ≤ p ≤ ∞, we denote by…”

“…Affine transformation-invariance, and particularly rotation-invariance, is an important data feature, prevalent in such areas as statistical physics [18], early warning of earthquakes [28], 3-D point-cloud segmentation [27], and image rendering [22]. Theoretically, neural networks with one hidden layer (to be called shallow nets) are incapable of embodying rotation-invariance features in the sense that its performance in handling these features is analogous to the failure of algebraic polynomials [13] in handling this task [14]. The primary goal of this paper is to construct neural networks with at least two hidden layers (called deep nets) to realize rotation-invariant features by deriving "fast" approximation and learning rates of radial functions as target functions.…”

The essential rate of approximation for radial function manifold

“…The results revealed that in order to approximate functions in the Sobolev space, radial functions manifolds and ridge functions manifolds have the same approximation capacity (see [12] and [13]). In [10], the upper and lower bounds of approximation by ridge function manifolds have been established in a larger Sobolev space than that of [12]. Thus it is natural to arise the question: does this upper bound and lower bound also hold for RFMs?…”