The first aim of this paper is to establish the weak convergence rate of nonlinear two-time-scale stochastic approximation algorithms. Its second aim is to introduce the averaging principle in the context of two-time-scale stochastic approximation algorithms. We first define the notion of asymptotic efficiency in this framework, then introduce the averaged two-time-scale stochastic approximation algorithm, and finally establish its weak convergence rate. We show, in particular, that both components of the averaged two-time-scale stochastic approximation algorithm simultaneously converge at the optimal rate √ n.

We apply the stochastic approximation method to construct a large class of recursive kernel estimators of a probability density, including the one introduced by Hall and Patil [1994. On the efficiency of on-line density estimators. IEEE Trans. Inform. Theory 40, 1504-1512]. We study the properties of these estimators and compare them with Rosenblatt's nonrecursive estimator. It turns out that, for pointwise estimation, it is preferable to use the nonrecursive Rosenblatt's kernel estimator rather than any recursive estimator. A contrario, for estimation by confidence intervals, it is better to use a recursive estimator rather than Rosenblatt's estimator.

No abstract

No abstract

No abstract

A stochastic algorithm for the recursive approximation of the location θ of a maximum of a regression function was introduced by Kiefer and Wolfowitz [Ann. Math. Statist. 23 (1952) 462-466] in the univariate framework, and by Blum [Ann. Math. Statist. 25 (1954) 737-744] in the multivariate case. The aim of this paper is to provide a companion algorithm to the Kiefer-Wolfowitz-Blum algorithm, which allows one to simultaneously recursively approximate the size µ of the maximum of the regression function. A precise study of the joint weak convergence rate of both algorithms is given; it turns out that, unlike the location of the maximum, the size of the maximum can be approximated by an algorithm which converges at the parametric rate. Moreover, averaging leads to an asymptotically efficient algorithm for the approximation of the couple (θ, µ).

No abstract

This paper studies the large deviations behaviour of the kernel estimator of a probability density f , by considering the case when the kernel takes negative values. It establishes large and moderate deviations principles for the kernel estimators of the partial derivatives of f . The estimators of the derivatives exhibit a quadratic behaviour for both the large and the moderate deviations scales, whereas for the density estimator there is a classical gap between the large deviations and the moderate deviations asymptotics.