Online Optimization of Dynamical Systems With Deep Learning Perception

Abstract

This paper considers the problem of controlling a dynamical system when the state cannot be directly measured and the control performance metrics are unknown or only partially known. In particular, we focus on the design of data-driven controllers to regulate a dynamical system to the solution of a constrained convex optimization problem where: i) the state must be estimated from nonlinear and possibly high-dimensional data; and ii) the cost of the optimization problem -which models control objectives associated with inputs and states of the system -is not available and must be learned from data. We propose a data-driven feedback controller that is based on adaptations of a projected gradient-flow method; the controller includes neural networks as integral components for the estimation of the unknown functions. Leveraging stability theory for perturbed systems, we derive sufficient conditions to guarantee exponential input-to-state stability (ISS) of the control loop. In particular, we show that the interconnected system is ISS with respect to the approximation errors of the neural network and unknown disturbances affecting the system. The transient bounds combine the universal approximation property of deep neural networks with the ISS characterization. Illustrative numerical results are presented in the context of robotics and control of epidemics.

Online Convex Optimization for Data-Driven Control of Dynamical Systems

“…Remark 1: As discussed above, our setting and hence also the regret bound from Theorem 1 is applicable to different application scenarios considered in the literature. For example, in perception-based control [25], [28], [29], v t is the error from the perception maps, which can be further bounded if, e.g., a residual neural network is used for perception [29]. More specifically, our results are applicable to the setting considered in [28], where a tracking control problem for an LTI system with complex, nonlinear measurements is studied (note that, as a special case, our results also hold for w t ≡ 0).…”

“…where where the state x t is estimated via perception maps [25], [29], and (iii) pseudo-measurements (as frequently employed in power systems [30], [31]), all of which result in state measurement noise v t . We have the following two standard assumptions on the disturbances and the system.…”

“…In this work, we propose a framework for robust control of dynamical systems subject to a priori unknown and timevarying cost functions, and state and input constraints that have to be met at each time instance. In particular, we consider disturbances acting on the system as well as measurement noise, which can capture, e.g., model mismatch, exogenous (uncontrollable) inputs to the system, sensor inaccuracies, state estimation error due to the application of an observer or perception-based techniques [25], [28], [29], and pseudomeasurement in the context of power systems [30], [31]. The combination of these types of disturbances with state and input constraints that have to be satisfied at all times has -to the best of the authors' knowledge -not been studied within the OCO framework, with the notable exception of [20].…”

Nonconvex Distributed Feedback Optimization for Aggregative Cooperative Robotics

“…In [16], a feedback optimization technique is designed for linear time-invariant systems. The approach is based on gradient flow dynamics augmented with learning methods to estimate the cost function based on infrequent and possibly noisy data.…”