On the Regret of H_∞ Control

Abstract

The H∞ synthesis approach is a cornerstone robust control design technique, but is known to be conservative in some cases. The objective of this paper is to quantify the additional cost the controller incurs planning for the worst-case scenario, by adopting an approach inspired by regret from online learning. We define the disturbance-reality gap as the difference between the predicted worst-case disturbance signal and the actual realization. The regret is shown to scale with the norm of this gap, which turns out to have a similar structure to that of the certainty equivalent controller with inaccurate predictions, obtained here in terms of the prediction error norm.

Online Linear Quadratic Tracking with Regret Guarantees

“…As J t (x, u ⋆ ) can be represented as an extended quadratic function of x, there exist f ∈ R m and g ∈ R [13] such that 15) is then a difference of two extended quadratic functions of the input, completing the proof.…”

“…The LQT problem can also be recast into an equivalent LQR formulation [8] by considering instead the dynamics…”

Distributed Online System Identification for LTI Systems Using Reverse Experience Replay

“…Stability: As mentioned previously, the resulting regret of an online control policy can be regarded as an indicator of the stability of the corresponding closed loop system. In the work of [1], it was shown that for linear policies and linear systems subject to adversarial noises, linear regret implies asymptotic stability for both time-invariant and time-varying systems. Conversely, the asymptotic stability (bounded input-bounded state stability and absolute summability of the state transition matrices, respectively) implies linear regret for time-invariant systems (time-varying systems, respectively).…”

“…In online control, the goal is to keep the regret bound sub-linear with respect to the time horizon, which implies that the time-averaged performance of the online controller mimics that of the best policy asymptotically. Though the regret measure originates from the online optimization framework, it has been shown that there exist direct connections between regret and the stability properties of linear and non-linear systems [1], [2] studied in control theory, which further justifies the study of regret for online control. For challenges posed in (I-II), recent works often transform the respective online control problem to an online learning and leverage online optimization techniques to design online controllers (e.g., semidefinite programming (SDP) relaxation for time-varying LQR [3] and noise feedback policy design for time-varying convex costs in [4]).…”

Robust regret optimal control

“…Subsequent work considered the multiplicative ratio in performance, called the competitive ratio, achieved by a given controller relative to the non‐causal controller 9,10 . Additional related work considers state‐and input constraints using system level synthesis 11 and regret‐bounds for $$$ {H}_{\infty } $$$ controllers 12 . A second thread measures regret of a given controller relative to the best static state‐feedback with full, non‐causal knowledge of the disturbance sequence 13 .…”