Nonlinear approximation using Gaussian kernels

Abstract

It is well known that nonlinear approximation has an advantage over linear schemes in the sense that it provides comparable approximation rates to those of the linear schemes, but to a larger class of approximands. This was established for spline approximations and for wavelet approximations, and more recently by DeVore and Ron (in press) [2] for homogeneous radial basis function (surface spline) approximations. However, no such results are known for the Gaussian function, the preferred kernel in machine learning and several engineering problems. We introduce and analyze in this paper a new algorithm for approximating functions using translates of Gaussian functions with varying tension parameters. At heart it employs the strategy for nonlinear approximation of DeVore-Ron, but it selects kernels by a method that is not straightforward. The crux of the difficulty lies in the necessity to vary the tension parameter in the Gaussian function spatially according to local information about the approximand: error analysis of Gaussian approximation schemes with varying tension are, by and large, an elusive target for approximators. We show that our algorithm is suitably optimal in the sense that it provides approximation rates similar to other established nonlinear methodologies like spline and wavelet approximations. As expected and desired, the approximation rates can be as high as needed and are essentially saturated only by the smoothness of the approximand.

Regular families of kernels for nonlinear approximation

“…Gaussians. This example is due to Hangelbroek and Ron and can be found in [26]. It served as the inspiration for our analysis.…”

“…While the proof of the main theorem is essentially the same as in [26], it depends on the propositions in Section 3 which, on account of conditions (A1)-(A6), allow for the use of kernels that have only finite smoothness.…”

Reactive molecular dynamics: From small molecules to proteins

“…where K R, R i ð Þ is a radial kernel 119,121 and G ϕ, ϕ j À Á is a Gaussian kernel. 184 The α λ, i, j are coefficients which follow from a singular value decomposition. 185,186 In parametrized form, the radial reproducing kernels are…”

Green’s Functions: Taking Another Look at Kernel Approximation, Radial Basis Functions, and Splines

“…Our convergence results pay special attention to the dependence of the estimates on the space dimension d. We will see that the use of anisotropic Gaussian kernels instead of isotropic ones provides improved convergence rates. It should also be mentioned that the work in [9] deals with linear approximation algorithms, while the recent paper [17] addresses nonlinear Gaussian approximation.…”