Stochastic Averaging in Discrete Time and its Applications to Extremum Seeking

Abstract

We investigate stochastic averaging theory for locally Lipschitz discrete-time nonlinear systems with stochastic perturbation and its applications to convergence analysis of discrete-time stochastic extremum seeking algorithms. Firstly, by defining two average systems (one is continuous time, the other is discrete time), we develop discretetime stochastic averaging theorem for locally Lipschitz nonlinear systems with stochastic perturbation. Our results only need some simple and applicable conditions, which are easy to verify, and remove a significant restriction present in existing results: global Lipschitzness of the nonlinear vector field. Secondly, we provide a discrete-time stochastic extremum seeking algorithm for a static map, in which measurement noise is considered and an ergodic discrete-time stochastic process is used as the excitation signal. Finally, for discrete-time nonlinear dynamical systems, in which the output equilibrium map has an extremum, we present a discrete-time stochastic extremum seeking scheme and, with a singular perturbation reduction, we prove the stability of the reduced system. Compared with classical stochastic approximation methods, while the convergence that we prove is in a weaker sense, the conditions of the algorithm are easy to verify and no requirements (e.g., boundedness) are imposed on the algorithm itself.

Extremum Seeking-based Adaptive PID Control applied to Neuromuscular Electrical Stimulation

“…For future research, an alternative to deterministic ES (Krstic and Wang 2000) is the stochastic ES method in (Liu and Krstic 2016). Hence, the papers Wang 2000, Liu andKrstic 2016) can be viewed as companion references for the practical user of the results in the highlighted paper. However, the user should be aware that, for nonlinear plants, optimal parameter choices will be dependent on the plant's initial condition and the setpoint value.…”

Model-Free Nonlinear Feedback Optimization

“…Traditional ES is mostly suitable for low-dimensional systems, for which the orthogonality requirement on the elements of the perturbation signal is easy to satisfy [3]. To handle high dimensionality, stochastic ES with perturbations governed by stochastic processes [4] and Newton-based ES [5], [6] have been developed. However, the convergence is established in the asymptotic sense, without characterization of its dependence on the problem dimension and stability properties of the plant.…”

Model‐based vs data‐driven adaptive control: An overview

“…Their basic goal is to seek an extremum, ie, maximize (or minimize), of a given function without closed‐from knowledge of the function or its gradient. There have been a lot of results on ES algorithms,) following the appearance of a rigorous convergence analysis in the work of Krstić and Wang …”