Solving the Infinite-Horizon Constrained LQR Problem Using Accelerated Dual Proximal Methods

Abstract

This work presents an algorithmic scheme for solving the infinite-time constrained linear quadratic regulation problem. We employ an accelerated version of a popular proximal gradient scheme, commonly known as the Forward-Backward Splitting (FBS), and prove its convergence to the optimal solution in our infinite-dimensional setting. Each iteration of the algorithm requires only finite memory, is computationally cheap, and makes no use of terminal invariant sets; hence, the algorithm can be applied to systems of very large dimensions. The acceleration brings in 'optimal' convergence rates O(1/k 2 ) for function values and O(1/k) for primal iterates and renders the proposed method a practical alternative to model predictive control schemes for setpoint tracking. In addition, for the case when the true system is subject to disturbances or modelling errors, we propose an efficient warm-starting procedure, which significantly reduces the number of iterations when the algorithm is applied in closed-loop. Numerical examples demonstrate the approach. Index TermsConstrained LQR, Alternating minimization, Operator splitting I. INTRODUCTION An important extension of the famous result of [1] on the closed form solution of the infinitehorizon linear quadratic regulation (LQR) problem is the case where input and state variables are constrained. This problem is computationally significantly more difficult and has been by and large addressed only approximately. A prime example of an approximation scheme is model predictive control (MPC), which approximates the infinite-time constrained problem by a finitetime one. Stability of such MPC controllers is then typically enforced by adding a suitable terminal constraint and a terminal penalty. The inclusion of a terminal constraint limits the feasible region of the MPC controller, and, consequently, the region of attraction of the closedloop system. In practical applications, this problem is typically overcome by simply choosing a "sufficiently" long horizon based on process insight (e.g., dominant time constant). Closed-loop behavior is then analyzed a posteriori, for instance by exhaustive simulation.There have been few results addressing directly the infinite-horizon constrained LQR (CLQR) problem. Among the most well-known efforts are the works

Fault‐tolerant reference generation for model predictive control with active diagnosis of elevator jamming faults

“…The terminal set associated with the augmented aircraft model takes into account also the dynamics of the actuators. Consequently, changes in the actuator bounds will impact the dynamics and the choice of the associated tightening parameters. An interesting alternative to be investigated (as part of our future research and out of the scope of this manuscript) is related to the use of infinite‐horizon MPC formulations,() which have been recently gaining increasing attention and can remove the requirements of a terminal set in the MPC problem formulation.…”

“…The robust terminal set for tracking computed based on the worst combination of faults can then be used in the MPC formulation (leading to a tube-based MPC design 43 for tracking). An interesting alternative to be investigated (as part of our future research and out of the scope of this manuscript) is related to the use of infinite-horizon MPC formulations, [44][45][46] which have been recently gaining increasing attention and can remove the requirements of a terminal set in the MPC problem formulation.…”

Constrained LQR using online decomposition techniques

“…Concerning λ s−1 N s , any value such that λ s−1 N s ∈ C (according to Lemmas 1 and 2) can be used (e.g., λ s−1 N s = λ s−1 N s −1 ). In [12] splitting strategies are also used to estimate N . Compared to [12], we use a different strategy to compute the length of the prediction horizon online.…”

“…Specifically, if our initial guess is too conservative, Algorithm 4 starts removing the tail subproblems. Removing subproblems implies removing dual variables that could, in general, affect the future updates of the algorithm, such as in [12]. This is not the case for the time splitting.…”

“…We show, under mild assumptions on the cost and on the system dynamics, that the length of the horizon is bounded and the solution using our algorithm is an optimal solution of the CLQR problem. Finally, numerical results using the planar system of [12] are provided to show the potential of our proposed approach.…”