Optimal pointwise adaptive methods in nonparametric estimation

Abstract

The problem of optimal adaptive estimation of a function at a given point from noisy data is considered. Two procedures are proved to be asymptotically optimal for different settings.First we study the problem of bandwidth selection for nonparametric pointwise kernel estimation with a given kernel. We propose a bandwidth selection procedure and prove its optimality in the asymptotic sense. Moreover, this optimality is stated not only among kernel estimators with a variable bandwidth. The resulting estimator is asymptotically optimal among all feasible estimators. The important feature of this procedure is that it is fully adaptive and it "works" for a very wide class of functions obeying a mild regularity restriction. With it the attainable accuracy of estimation depends on the function itself and is expressed in terms of the "ideal adaptive bandwidth" corresponding to this function and a given kernel.The second procedure can be considered as a specialization of the first one under the qualitative assumption that the function to be estimated belongs to some Hölder class β L with unknown parameters β L. This assumption allows us to choose a family of kernels in an optimal way and the resulting procedure appears to be asymptotically optimal in the adaptive sense in any range of adaptation with β ≤ 2.

Nonlinear system identification via direct weight optimization

“…This might deteriorate the performance of the estimation methods. To a certain extent, this problem can be compensated for by choosing the bandwidth in an adaptive way (see, e.g., Lepski and Spokoiny, 1997). However, the shape of the kernels is still fixed and is not adjusted to the actual data.…”

Exact adaptive pointwise estimation on Sobolev classes of densities

“…Finally, a major difference with respect to the nonparametric regression or Gaussian white noise model, as considered in Lepski and Spokoiny [28], Tsybakov [30], is the fact that the density model is heteroscedastic. This means more precisely that the variance of the kernel estimator is proportional to f (x 0 ), the value of the unknown density at the estimation point.…”

“…For further developments see Golubev [14,15], Golubev and Nussbaum [17]. Then Lepskii [23], Lepski and Spokoiny [28] obtained exact adaptive results in L ∞ and at a fixed point, respectively, on the Hölder classes with 0 < β ≤ 2 (see also Lepskii [24] and [25]). Tsybakov [30] proved exact adaptive results for the Gaussian white noise model both in L ∞ and at a fixed point, on the Sobolev classes.…”

Multiresolution local polynomial regression: A new approach to pointwise spatial adaptation

“…A recent break-through in pointwise varying scale estimation adaptive to unknown smoothness of the function is originated from a general scheme of Lepski [28,29,38] already mentioned in the introduction. The LPA estimates are calculated for a grid of scales and compared.…”

“…The idea of the used Lepski's adaptation method is as follows, [27][28][29]38]. The algorithm searches for a largest local vicinity of the point of estimation where the LPA assumption fits well to the data.…”