Noise conditions for prespecified convergence rates of stochastic approximation algorithms

Abstract

Abstract-We develop deterministic necessary and sufficient conditions on individual noise sequences of a stochastic approximation algorithm for the error of the iterates to converge at a given rate.Specifically, suppose f n g f n g f n g is a given positive sequence converging monotonically to zero. Consider a stochastic approximation algorithm x n+1 = x n 0 a n (A n x n 0 b n ) + a n e n x n+1 = x n 0 a n (A n x n 0 b n ) + a n e n x n+1 = x n 0 a n (A n x n 0 b n ) + a n e n , where fx n g fx n g fx n g is the iterate sequence, fang fang fang is the step size sequence, feng feng feng is the noise sequence, and x 3 x 3x 3 is the desired zero of the function f (x) = Ax 0 b f (x) = Ax 0 b f (x) = Ax 0 b. Then, under appropriate assumptions, we show that x n 0 x 3 = o( n )x n 0 x 3 = o( n ) x n 0 x 3 = o( n ) if and only if the sequence feng feng feng satisfies one of five equivalent conditions. These conditions are based on well-known formulas for noise sequences: Kushner and Clark's condition, Chen's condition, Kulkarni and Horn's condition, a decomposition condition, and a weighted averaging condition. Our necessary and sufficient condition on feng feng feng to achieve a convergence rate of f n g f n g f n g is basically that the sequence fe n = n g fe n = n g fe n = n g satisfies any one of the above five well-known conditions. We provide examples to illustrate our result. In particular, we easily recover the familiar result that if a n = a=n a n = a=n a n = a=n and fe n g fe n g fe n g is a martingale difference process with bounded variance, then xn 0 x 3 = o(n xn 0 x 3 = o(n xn 0 x 3 = o(n 01=2 (log (n)) ) (log (n)) ) (log (n)) )for any > > >1=2.Index Terms-Convergence rate, Kiefer-Wolfowitz, necessary and sufficient conditions, noise sequences, Robbins-Monro, stochastic approximation.

Linear Stochastic Approximation Algorithms and Group Consensus Over Random Signed Networks

“…From Theorem 3.1, x(s) converges to a random vector under the critical condition ρ max (P ) = 1 (or ρ min (P ) = 1), which is different from the previous works where x(s) converges to a deterministic vector under non critical conditions [6]- [8], [19], [28], [39]. Due to this difference, the traditional method cannot be used in the proof of Theorem 3.1.…”

“…For {P (s)} and {u(s)}, we relax the i.i.d. condition in [34] to the following assumption: Under the assumptions (A1) and (A2), the previous works has investigated the cases when ρ max (P ) < 1 and P (s)x + u(s) is a bounded linear operator for all s ≥ 0 [7], [8], or ρ max (P + αI n ) < 1 [39], or {P (s)} are row-stochastic matrices and u = 0 [6], [19], [28]. This paper will consider all the cases of P and u, and show the necessary and sufficient condition for the convergence of x(s) in system (2) is ρ max (P ) < 1, or ρ max (P ) = 1 together with the following condition for P and u:…”

“…, (2) where a(s) ∈ R is the gain function. The system (1) is so called as linear SA algorithms [6]- [8], [19], [28], [39]. Compared to system (1), each agent in system (1) updates its state depending not only on the linear map P (s)x(s) + u(s) but also on its own current state.…”

“…A main thread in the research of such a system is to study the setting in which x(s) converges to a deterministic point. In [7], [8], convergence and convergence rates are studied for bounded linear operators with the assumption that there exists a matrix P ∈ R n×n whose eigenvalues' real parts are all less than 1 such that…”

“…Currently, there are two main threads on the theoretical research of linear SA algorithms. One thread is based on assumptions that guarantee the state of the system converges to a deterministic point [7], [8], [24], [25], [39]. Another thread is the research on consensus of multi-agent systems, where the system matrices are assumed to be row-stochastic [6], [19], [28].…”