Robust Control of Unknown Observable Nonlinear Systems Solved as a Zero-Sum Game

Abstract

An optimal robust control solution for general nonlinear systems with unknown but observable dynamics is advanced here. The underlying Hamilton-Jacobi-Isaacs (HJI) equation of the corresponding zero-sum two-player game (ZS-TP-G) is learned using a Q-learning-based approach employing only input-output system measurements, assuming system observability. An equivalent virtual state-space model is built from the system's input-output samples and it is shown that controlling the former implies controlling the latter. Since the existence of a saddle-point solution to the ZS-TP-G is assumed unverifiable, the solution is derived in terms of upper-optimal and lower-optimal controllers. The learning convergence is theoretically ensured while practical implementation is performed using neural networks that provide scalability to the control problem dimension and automatic feature selection. The learning strategy is checked on an active suspension system, a good candidate for the robust control problem with respect to road profile disturbance rejection. INDEX TERMS active suspension system, approximate dynamic programming, neural networks, optimal control, reinforcement learning, state feedback, zero-sum two-player games

Virtual State Feedback Reference Tuning and Value Iteration Reinforcement Learning for Unknown Observable Systems Control

“…are employed to form a virtual state-space model having k u as input and k y as output (similarly to (1)), with a different state vector, according to [45]. This resulted virtual state-space model is:…”

“…(upper T means vector/matrix transposition). Model (2) has partially known dynamics (the unknown part stems from unknown dynamics of (1)) and it is fully state observable [45]. Such transformations are well-known for linear systems.…”

“…Here, τ is correlated with the observability index and should be chosen empirically since it cannot be established analytically due to partially unknown dynamics. It should be selected as large as possible, accounting for the fact that for a value larger than the true observability index, there is no more information gain in explaining the true state k s through k v [45]. The observability index is wellknown in linear systems theory (please consult Appendix A).…”

“…are CD since they emerge from the CD map F in (2), whose CD is a consequence of the CD property of f, g from (1) [45]. Their bounded derivatives are reasonable assumptions for that matter.…”

“…Transformations such as (2) can easily accommodate time delays in the input/state of (1) by properly introducing supplementary states. Such operations preserve the full observability of (2) [45]. IO controlling (2) is the same with IO controlling (1), since they have the same input and output.…”