Analysis of Target Data-Dependent Greedy Kernel Algorithms: Convergence Rates for f-, $$f \cdot P$$- and f/P-Greedy

Abstract

Data-dependent greedy algorithms in kernel spaces are known to provide fast converging interpolants, while being extremely easy to implement and efficient to run. Despite this experimental evidence, no detailed theory has yet been presented. This situation is unsatisfactory, especially when compared to the case of the data-independent P-greedy algorithm, for which optimal convergence rates are available, despite its performances being usually inferior to the ones of target data-dependent algorithms. In this work, we fill this gap by first defining a new scale of greedy algorithms for interpolation that comprises all the existing ones in a unique analysis, where the degree of dependency of the selection criterion on the functional data is quantified by a real parameter. We then prove new convergence rates where this degree is taken into account, and we show that, possibly up to a logarithmic factor, target data-dependent selection strategies provide faster convergence. In particular, for the first time we obtain convergence rates for target data adaptive interpolation that are faster than the ones given by uniform points, without the need of any special assumption on the target function. These results are made possible by refining an earlier analysis of greedy algorithms in general Hilbert spaces. The rates are confirmed by a number of numerical examples.

Data-driven kernel designs for optimized greedy schemes: A machine learning perspective

“…Due to its small expansion size, the greedy interpolant s n can be understood as a sparse approximation of s X N . An established way for achieve this in the context of surrogate modeling is to select a meaningful subset X n ⊂ X N of the training data X N via greedy kernel methods [45,48]. These are iterative schemes that start with an empty set X 0 = {}.…”

“…In the following Subsection 2.1 we review the basics of kernel methods [13,42], while in the subsequent Subsection 2.2 we briefly introduce greedy kernel methods [45,48].…”

“…As far as the first item is concerned, assuming that a set of measurements sampled at multivariate scattered points is given, the basic idea of greedy kernel methods [45,48] consists in selecting only a smaller subset (called centers) out of the large training set of scattered points to build the kernel model. For this, one starts with an empty set of centers and then adds one more center, step by step, according to some selection criterion [9,11,30,39], until a predefined amount of centers or some other stopping criterion is met.…”

Stable interpolation with exponential-polynomial splines and node selection via greedy algorithms

“…We would like to point out that the notation and terminology used in this section is taken from the literature on greedy kernel methods (see, e.g., [27]), where a similar distinction has been introduced for the target-data-independent P-greedy algorithm, and the target-data-dependent f-, f/P-, and f ⋅ P-greedy algorithms.…”

“…However, since these uniform sampling locations may not be available in practical applications, we further consider incremental methods that, given an initial set of samples, construct an EPS interpolant by iteratively selecting a new point at each iteration. The iterative rule is dictated by greedy methods (see [21]), which have been investigated, e.g., for kernel interpolation (refer, e.g., to [22][23][24][25][26][27]) and lead to sparse models which turn out to be helpful in many applications, see, e.g., [28]. This iterative selection is a convenient proxy for the optimal selection of the sampling points from a fixed set, which is in turn usually an extremely computationally demanding procedure.…”