No abstract

No abstract

Several problems in modeling and control of stochastically-driven dynamical systems can be cast as regularized semi-definite programs. We examine two such representative problems and show that they can be formulated in a similar manner. The first, in statistical modeling, seeks to reconcile observed statistics by suitably and minimally perturbing prior dynamics. The second seeks to optimally select a subset of available sensors and actuators for control purposes. To address modeling and control of large-scale systems we develop a unified algorithmic framework using proximal methods. Our customized algorithms exploit problem structure and allow handling statistical modeling, as well as sensor and actuator selection, for substantially larger scales than what is amenable to current general-purpose solvers. We establish linear convergence of the proximal gradient algorithm, draw contrast between the proposed proximal algorithms and alternating direction method of multipliers, and provide examples that illustrate the merits and effectiveness of our framework. Index TermsActuator selection, sensor selection, sparsity-promoting estimation and control, method of multipliers, nonsmooth convex optimization, proximal algorithms, regularization for design, semi-definite programming, structured covariances. I. INTRODUCTIONConvex optimization has had tremendous impact on many disciplines, including system identification and control design [1]- [7]. The forefront of research points to broadening the range of applications as well as sharpening the effectiveness of algorithms in terms of speed and scalability. The present paper focuses on two representative control problems, statistical control-oriented modeling and sensor/actuator selection, that are cast as convex programs. A range of modern applications require addressing these over increasingly large parameter spaces, placing them outside the reach of standard solvers. A contribution of the paper is to formulate such problems as regularized semi-definite programs (SDPs) and to develop customized optimization algorithms that scale favorably with size.Modeling is often seen as an inverse problem where a search in parameter space aims to find a parsimonious representation of data. For example, in the control-oriented modeling of fluid flows, it is of interest to improve upon dynamical equations arising from first-principles (e.g., linearized Navier-Stokes equations), in order to accurately 2 replicate observed statistical features that are estimated from data. To this end, a perturbation of the prior model can be seen as a feedback gain that results in dynamical coupling between a suitable subset of parameters [8], [9].On the flip side, active control of large-scale and distributed systems requires judicious placement of sensors and actuators which again can be viewed as the selection of a suitable feedback or Kalman gain. In either modeling or control, the selection of such gain matrices must be guided by optimality criteria as well as simplicity (low rank or sparse architectur...

Model-free reinforcement learning attempts to find an optimal control action for an unknown dynamical system by directly searching over the parameter space of controllers. The convergence behavior and statistical properties of these approaches are often poorly understood because of the nonconvex nature of the underlying optimization problems as well as the lack of exact gradient computation. In this paper, we take a step towards demystifying the performance and efficiency of such methods by focusing on the standard infinite-horizon linear quadratic regulator problem for continuous-time systems with unknown state-space parameters. We establish exponential stability for the ordinary differential equation (ODE) that governs the gradient-flow dynamics over the set of stabilizing feedback gains and show that a similar result holds for the gradient descent method that arises from the forward Euler discretization of the corresponding ODE. We also provide theoretical bounds on the convergence rate and sample complexity of a random search method. Our results demonstrate that the required simulation time for achieving -accuracy in a model-free setup and the total number of function evaluations both scale as log (1/ ).

Optimization algorithms are increasingly being used in applications with limited time budgets. In many real-time and embedded scenarios, only a few iterations can be performed and traditional convergence metrics cannot be used to evaluate performance in these non-asymptotic regimes. In this paper, we examine the transient behavior of accelerated first-order optimization algorithms. For quadratic optimization problems, we employ tools from linear systems theory to show that transient growth arises from the presence of non-normal dynamics. We identify the existence of modes that yield an algebraic growth in early iterations and quantify the transient excursion from the optimal solution caused by these modes. For strongly convex smooth optimization problems, we utilize the theory of integral quadratic constraints to establish an upper bound on the magnitude of the transient response of Nesterov's accelerated method. We show that both the Euclidean distance between the optimization variable and the global minimizer and the rise time to the transient peak are proportional to the square root of the condition number of the problem. Finally, for problems with large condition numbers, we demonstrate tightness of the bounds that we derive up to constant factors.

No abstract

No abstract

No abstract