Online Stochastic Optimization for Unknown Linear Systems: Data-Driven Synthesis and Controller Analysis

Abstract

This paper proposes a data-driven control framework to regulate an unknown, stochastic linear dynamical system to the solution of a (stochastic) convex optimization problem. Despite the centrality of this problem, most of the available methods critically rely on a precise knowledge of the system dynamics (thus requiring off-line system identification and model refinement). To this aim, in this paper we first show that the steady-state transfer function of a linear system can be computed directly from control experiments, bypassing explicit model identification. Then, we leverage the estimated transfer function to design a controller -which is inspired by stochastic gradient descent methods -that regulates the system to the solution of the prescribed optimization problem. A distinguishing feature of our methods is that they do not require any knowledge of the system dynamics, disturbance terms, or their distributions. Our technical analysis combines concepts and tools from behavioral system theory, stochastic optimization with decision-dependent distributions, and stability analysis. We illustrate the applicability of the framework on a case study for mobility-on-demand ride service scheduling in Manhattan, NY.

Model-Free Nonlinear Feedback Optimization

“…Some works [20], [22], [23], [26] consider fast-stable plants that are abstracted as algebraic steady-state maps. Others take system dynamics into account and characterize sufficient conditions for the closedloop stability, including in continuous-time [21], [27]- [29] and sampled-data settings [30]. Specifically, among works that handle nonconvex objectives and nonlinear systems, [22], [26] address discrete-time systems represented by algebraic maps, while [28] tackles continuous-time systems.…”

“…Given objective functions depending on steady-state inputs and outputs, due to the chain rule, their gradients with respect to inputs naturally contain the sensitivity terms. Recent works demonstrate that the sensitivity can be estimated based on system identification [19], recursive least-squares estimation [32], or data-driven methods that use past input-output data of openloop linear systems [29], [33]. Nevertheless, on the one hand, the estimation can be a highly nontrivial task accompanied with errors, thus causing the approximate optimality of FO [18].…”

Stochastic Saddle Point Problems with Decision-Dependent Distributions

“…Uncertainty may be due to several factors, depending on the particular application domain and problem formulations. For example, in statistical learning, parameters may be taken to be data-label pairs in large data-sets [15,23]; in the context of optimization of physical and dynamical systems, they may model externalities and random exogenous inputs, or system parameters that are predicted from data and are accompanied by given error statistics [9]. A key assumption that is typically leveraged for providing theoretical guarantees for stochastic optimization algorithms is that the distributions of random parameters are stationary [10].…”

Online convex optimization for data-driven control of dynamical systems

“…Again, the main focus in the literature is on model-based control with process noise and output feedback [3], [25], [26]. Most relevant to this work is [27], where output feedback and unknown systems are treated by application of the fundamental lemma. However, the cost functions are assumed to be constant and time-variability of the optimization problem is only introduced via process noise.…”

“…Third, we consider noise in the measurement process. In the relevant literature, e.g., [9], [11], [27], the main research focus is on systems subject to process noise, which is typically handled by estimating the process noise using exact measurements and model knowledge. We instead consider only noisy measurements in our theoretical work and leave the combination of both, process and measurement noise, as an interesting topic for future research.…”