On Persistency of Excitation and Formulas for Data-driven Control

Persis, Claudio De; Tesi, Pietro

doi:10.1109/cdc40024.2019.9029185

Cited by 30 publications

(30 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1 To simplify the technical treatment and without compromising generality, we assume that x f is reachable in T -steps, i.e., (x f − A T x 0 ) ∈ Im(C T ). 2 While the full state trajectory could be measured [13], here we show that measuring the final state is sufficient to compute minimum-energy inputs.…”

Section: Learning Minimum-energy Control Inputsmentioning

confidence: 79%

Data-Driven Minimum-Energy Controls for Linear Systems

Baggio

Katewa

Pasqualetti

2019

IEEE Control Syst. Lett.

View full text Add to dashboard Cite

In this paper we study the problem of computing minimum-energy controls for linear systems from experimental data. The design of open-loop minimum-energy control inputs to steer a linear system between two different states in finite time is a classic problem in control theory, whose solution can be computed in closed form using the system matrices and its controllability Gramian. Yet, the computation of these inputs is known to be ill-conditioned, especially when the system is large, the control horizon long, and the system model uncertain. Due to these limitations, open-loop minimum-energy controls and the associated state trajectories have remained primarily of theoretical value. Surprisingly, in this paper we show that open-loop minimum-energy controls can be learned exactly from experimental data, with a finite number of control experiments over the same time horizon, without knowledge or estimation of the system model, and with an algorithm that is significantly more reliable than the direct model-based computation. These findings promote a new philosophy of controlling large, uncertain, linear systems where data is abundantly available.

show abstract

Section: Learning Minimum-energy Control Inputsmentioning

confidence: 79%

Data-Driven Minimum-Energy Controls for Linear Systems

Baggio

Katewa

Pasqualetti

2019

IEEE Control Syst. Lett.

View full text Add to dashboard Cite

show abstract

“…There remain fundamental open problems for learning in LQR problems, with several addressed only recently, including nonasymptotic sample complexity [4], [10], regret bounds [8], [11], [13], and algorithmic convergence [5]. Alternatives to reinforcement learning include other data-driven model-free optimal control schemes [38], [39] and those leveraging the behavioral framework [40], [41]. Subspace identification methods offer a model-based generalization to the output feedback setting [42].…”

Section: A Related Literaturementioning

confidence: 99%

Learning Optimal Controllers for Linear Systems With Multiplicative Noise via Policy Gradient

Gravell

Esfahani

Summers

2021

IEEE Trans. Automat. Contr.

View full text Add to dashboard Cite

The linear quadratic regulator (LQR) problem has reemerged as an important theoretical benchmark for reinforcement learning-based control of complex dynamical systems with continuous state and action spaces. In contrast with nearly all recent work in this area, we consider multiplicative noise models, which are increasingly relevant because they explicitly incorporate inherent uncertainty and variation in the system dynamics and thereby improve robustness properties of the controller. Robustness is a critical and poorly understood issue in reinforcement learning; existing methods which do not account for uncertainty can converge to fragile policies or fail to converge at all. Additionally, intentional injection of multiplicative noise into learning algorithms can enhance robustness of policies, as observed in ad hoc work on domain randomization. Although policy gradient algorithms require optimization of a nonconvex cost function, we show that the multiplicative noise LQR cost has a special property called gradient domination, which is exploited to prove global convergence of policy gradient algorithms to the globally optimum control policy with polynomial dependence on problem parameters. Results are provided both in the model-known and model-unknown settings where samples of system trajectories are used to estimate policy gradients

show abstract

“…Our goal is to control an unknown system using only the privacy-sensitive inputoutput data that are potentially noisy. Data-driven control is a blooming research area and methods based on the behavioral framework have received significant renewed interest [4]- [10] since their original proposal [11], [12]. The idea of such methods is that the state representation can be replaced by a data-based representation which only uses the trajectories of the system, bypassing the need for system identification.…”

Section: A Contributionsmentioning

confidence: 99%

“…The goal is to compute a reference-tracking LQR control, based only on precollected input-output data. We can reformulate this control problem inspired by the behavioral framework [4]- [6], [11].…”

Section: Case Study: Data Predictive Controlmentioning

confidence: 99%

Encrypted Distributed Lasso for Sparse Data Predictive Control

Alexandru¹,

Tsiamis²,

Pappas³

2021

Preprint

View full text Add to dashboard Cite

The least squares problem with 1-regularized regressors, called Lasso, is a widely used approach in optimization problems where sparsity of the regressors is desired. This formulation is fundamental for many applications in signal processing, machine learning and control. As a motivating problem, we investigate a sparse data predictive control problem, run at a cloud service to control a system with unknown model, using 1-regularization to limit the behavior complexity. The inputoutput data collected for the system is privacy-sensitive, hence, we design a privacy-preserving solution using homomorphically encrypted data. The main challenges are the non-smoothness of the 1-norm, which is difficult to evaluate on encrypted data, as well as the iterative nature of the Lasso problem. We use a distributed ADMM formulation that enables us to exchange substantial local computation for little communication between multiple servers. We first give an encrypted multi-party protocol for solving the distributed Lasso problem, by approximating the non-smooth part with a Chebyshev polynomial, evaluating it on encrypted data, and using a more cost effective distributed bootstrapping operation. For the example of data predictive control, we prefer a non-homogeneous splitting of the data for better convergence. We give an encrypted multi-party protocol for this non-homogeneous splitting of the Lasso problem to a non-homogeneous set of servers: one powerful server and a few less powerful devices, added for security reasons. Finally, we provide numerical results for our proposed solutions.

show abstract

On Persistency of Excitation and Formulas for Data-driven Control

Cited by 30 publications

References 32 publications

Data-Driven Minimum-Energy Controls for Linear Systems

Data-Driven Minimum-Energy Controls for Linear Systems

Learning Optimal Controllers for Linear Systems With Multiplicative Noise via Policy Gradient

Encrypted Distributed Lasso for Sparse Data Predictive Control

Contact Info

Product

Resources

About