A result on the mean square error obtained using general tracking algorithms

Abstract

Tracking time-varying properties is of crucial importance in all adaptive algorithms. In this contribution we study a fairly general algorithm for tracking properties of model parameters that can be described in a linear regression form (including AR models and the like). An explicit expression for the mean square error between the estimated and the true (time-varying) parameter is established. For slow adaptation this expression can be arbitrarily well approximated by a much simpler expression. The treatment differs from other related studies using weak convergence theory, averaging, etc. in that the results are not asymptotic in nature and are applicable also to the transient phase as well as over unbounded time intervals.

Necessary and sufficient conditions for stability of LMS

“…Theorem 2: Consider (2). Let the signal process be generated by (14) where is a bounded deterministic sequence, and is a -mixing process which satisfies for any and any integer sequence (15) where and are positive constants. Then for any there exist constants and such that for all (16) if and only if there exists an integer and a constant such that (17) The proof is also given in Section IV.…”

A robust algorithm for random parameter tracking

“…Theorem 3 supplies bounds on the tracking error without a "weak dependence" assumption on tut: if wt is not a centered variable, then the asymptotic error bound deteriorates (it includes a term of the form Jr; + r / p instead of Jr; + r/,/ii as in Theorem 4, cf. Theorem 3 in [15]). If we suppress also the condition on It, which implies implicitly a weaker assumption on et, then further degradation takes place.…”

“…Moreover, there are some q 2 4 and C2(q) < 00 such that for any t 2 1 00 CIIE(cpicpT -DilFtTt)ll, I C2(4). (15) Remark: Actually we only use the existence of the moments v;* < m, E ; . < 00 and w;* < 00 for some large l* which depends on q in (15) (cf.…”

Stochastic approximation with nondecaying gain: Error bound and data‐driven gain‐tuning

“…Given that the sum of the two real roots is 4(r 2 k − 1) > 0, and the product of the two real roots is 4(1 − r 2 k ) < 0, we know that the smaller root is negative and the larger root q k,1 is positive. Therefore, when r k > 1, 4 k (q + 2) 2 − 4ℒ 2 k 2 k (q + 1) ≤ 0 is nonpositive for any q satisfying (11). ▪…”