Structured Output Feedback Control for Linear Quadratic Regulator Using Policy Gradient Method

Abstract

We consider the static output feedback control for Linear Quadratic Regulator problems with the feedback gain constraints from the perspective of the reinforcement learning. In the model based setting, we introduce the gradient projection method and show its global convergence to stationary points. To solve the problem in the model free setting, we propose the policy gradient algorithm based on the gradient projection method and show its global convergence to ε-stationary points. In addition, we introduce a variance reduction technique and show both theoretically and numerically that it significantly reduces the variance in the gradient estimation. We also show in the numerical experiments that the model free approach performs almost as well as the model based approach.