A System Level Approach to Controller Synthesis

Wang, Yuh-Shyang; Matni, Nikolai; Doyle, John C.

doi:10.48550/arxiv.1610.04815

Cited by 16 publications

(47 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although classic methods exist for computing such controllers [22,46,53,60], they typically require solving nonconvex optimization problems, and it is not readily obvious how to extract interpretable measures of controller performance as a function of the perturbation sizes A and B . To that end, we leverage the recently developed System Level Synthesis (SLS) framework [59] to create an alternative robust synthesis procedure. Described in detail in Section 3, SLS lifts the system description into a higher dimensional space that enables efficient search for controllers.…”

Section: Problem Statement and Our Contributionsmentioning

confidence: 99%

“…To circumvent this issue, we leverage a novel parameterization of robustly stabilizing controllers based on the SLS framework for controller synthesis [59]. We describe this framework in more detail in Section 3.…”

Section: Related Workmentioning

confidence: 99%

“…In the case of LQR, step 1 of coarse-ID control simply requires solving a linear least squares problem, step 2 uses a finite sample theoretical guarantee or a standard bootstrap technique, and step 3 requires solving a small semidefinite program. Analyzing this approach, on the other hand, requires contemporary techniques in non-asymptotic statistics and a novel parameterization of control problems that renders nonconvex problems convex [39,59].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

On the Sample Complexity of the Linear Quadratic Regulator

et al. 2019

Self Cite

View full text Add to dashboard Cite

This paper addresses the optimal control problem known as the Linear Quadratic Regulator in the case when the dynamics are unknown. We propose a multi-stage procedure, called Coarse-ID control, that estimates a model from a few experimental trials, estimates the error in that model with respect to the truth, and then designs a controller using both the model and uncertainty estimate. Our technique uses contemporary tools from random matrix theory to bound the error in the estimation procedure. We also employ a recently developed approach to control synthesis called System Level Synthesis that enables robust control design by solving a quasiconvex optimization problem. We provide end-to-end bounds on the relative error in control cost that are optimal in the number of parameters and that highlight salient properties of the system to be controlled such as closed-loop sensitivity and optimal control magnitude. We show experimentally that the Coarse-ID approach enables efficient computation of a stabilizing controller in regimes where simple control schemes that do not take the model uncertainty into account fail to stabilize the true system.

show abstract

Section: Problem Statement and Our Contributionsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

On the Sample Complexity of the Linear Quadratic Regulator

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…In order to meet these goals, we use the System Level Synthesis [116], [117] (SLS) nominal and robust [118] parameterizations of stabilizing controllers. The SLS framework focuses on the system responses of a closed-loop system.…”

Section: A Pac Bounds For Unknown (A B)mentioning

confidence: 99%

From self-tuning regulators to reinforcement learning and back again

Matni

Proutière

Rantzer

et al. 2019

2019 IEEE 58th Conference on Decision and Control (CDC)

Self Cite

View full text Add to dashboard Cite

Machine and reinforcement learning (RL) are being applied to plan and control the behavior of autonomous systems interacting with the physical world -examples include self-driving vehicles, distributed sensor networks, and agile robots. However, if machine learning is to be applied in these new settings, the resulting algorithms must come with the reliability, robustness, and safety guarantees that are hallmarks of the control theory literature, as failures could be catastrophic. Thus, as RL algorithms are increasingly and more aggressively deployed in safety critical settings, it is imperative that control theorists be part of the conversation. The goal of this tutorial paper is to provide a jumping off point for control theorists wishing to work on RL related problems by covering recent advances in bridging learning and control theory, and by placing these results within the appropriate historical context of the system identification and adaptive control literatures.• Section II: provides an extensive literature review of work spanning classical and modern results in system identification, adaptive control, and RL.• Section III: introduces the fundamental problem and performance metrics considered in RL, and relates them to examples familiar to the controls community.• Section IV: provides a survey of contemporary results for problems with finite state and action spaces. • Section V: shows how system estimates and error bounds can be incorporated into model-based self-tuning regulators with finite-time performance guarantees.• Section VI: presents guarantees for model-free methods, and shows that a complexity gap exists between model-based and model-free methods. II. LITERATURE REVIEWThe results we present in this paper draw heavily from three broad areas of control and learning theory: system identification, adaptive control, and approximate dynamic programming (ADP) or, as it has come to be known, reinforcement learning. Each of these areas has a long and rich history and a general literature review is outside the scope of this tutorial. Below we will instead emphasize pointers to good textbooks and survey papers, before giving a more careful account of recent work.1) System Identification: The estimation of system behavior from input/output experiments has a well-developed theory dating back to the 1960s, particularly in the case of linear-time-invariant systems. Standard reference texts on the topic include [6], [8], [9], [10]. The success of discrete time series analysis by Box and Jenkins [11] provided an early impetus for the extension of these methods to the controlled system setting. Important connections to information theory were established by Akaike [12]. The rise of robust control in the 1980s further inspired system identification procedures, wherein model errors were optimized under the assumption of adversarial noise processes [13]. Another important step was the development of subspace methods [14], which became a powerful tool for identification of multi-input multi-output systems.2) Adaptive...

show abstract

“…When the system is spatially invariant [11], hierarchical [12], positive [13], or quadratic invariant [14], ODC has a convex formulation. A System Level Approach [15] also convexifies ODC at the expense of working with a series of impulse response matrices. Various approximation [16], [17], [18] and convex relaxation techniques [19], [20], [21] also exist in the literature.…”

Section: Introductionmentioning

confidence: 99%

Aggressive Local Search for Constrained Optimal Control Problems with Many Local Minima

Ding¹,

Han²,

Lavaei³

2019

Preprint

View full text Add to dashboard Cite

This paper is concerned with numerically finding a global solution of constrained optimal control problems with many local minima. The focus is on the optimal decentralized control (ODC) problem, whose feasible set is recently shown to have an exponential number of connected components and consequently an exponential number of local minima. The rich literature of numerical algorithms for nonlinear optimization suggests that if a local search algorithm is initialized in an arbitrary connected component of the feasible set, it would search only within that component and find a stationary point there. This is based on the fact that numerical algorithms are designed to generate a sequence of points (via searching for descent directions and adjusting the step size), whose corresponding continuous path is trapped in a single connected component. In contrast with this perception rooted in convex optimization, we numerically illustrate that local search methods for non-convex constrained optimization can obliviously jump between different connected components to converge to a global minimum, via an aggressive step size adjustment using backtracking and the Armijio rule. To support the observations, we prove that from almost every arbitrary point in any connected component of the feasible set, it is possible to generate a sequence of points using local search to jump to different components and converge to a global solution. However, due to the NP-hardness of the problem, such fine-tuning of the parameters of a local search algorithm may need prior knowledge or be time consuming. This paper offers the first result on escaping non-global local solutions of constrained optimal control problems with complicated feasible sets.

show abstract

A System Level Approach to Controller Synthesis

Cited by 16 publications

References 46 publications

On the Sample Complexity of the Linear Quadratic Regulator

On the Sample Complexity of the Linear Quadratic Regulator

From self-tuning regulators to reinforcement learning and back again

Aggressive Local Search for Constrained Optimal Control Problems with Many Local Minima

Contact Info

Product

Resources

About