This paper presents an online learning algorithm based on integral reinforcement learning (IRL) to design an output-feedback (OPFB) H ∞ tracking controller for partially unknown linear continuous-time systems. Although reinforcement learning techniques have been successfully applied to find optimal state-feedback controllers, in most control applications, it is not practical to measure the full system states. Therefore, it is desired to design OPFB controllers.To this end, a general bounded L 2 -gain tracking problem with a discounted performance function is used for the OPFB H ∞ tracking. A tracking game algebraic Riccati equation is then developed that gives a Nash equilibrium solution to the associated min-max optimization problem. An IRL algorithm is then developed to solve the game algebraic Riccati equation online without requiring complete knowledge of the system dynamics. The proposed IRL-based algorithm solves an IRL Bellman equation in each iteration online in real time to evaluate an OPFB policy and updates the OPFB gain using the information given by the evaluated policy. An adaptive observer is used to provide the knowledge of the full states for the IRL Bellman equation during learning. However, the observer is not needed after the learning process is finished. A simulation example is provided to verify the convergence of the proposed algorithm to a suboptimal OPFB solution and the performance of the proposed method.
KEYWORDSbounded L 2 -gain, H ∞ controller, optimal control, output feedback, reinforcement learning (RL)
INTRODUCTIONDesign of output-feedback (OPFB) controllers, particularly static OPFB, 1 is of paramount importance for many applications, such as aircraft control, 2 optimal process control, 3 and elsewhere, since full-state measurement is not usually possible in practical systems. Moreover, the use of OPFB for some tracking control problems allows flexibility and simplicity of implementation.Optimal tracking control problems seek to find a controller that not only stabilizes the tracking error dynamics but also optimizes its transient performance. This is achieved by minimizing a predefined performance index. For linear systems accompanied by a quadratic performance index, the optimal tracking problem is called the linear quadratic tracking (LQT) problem, which is an important problem in the field of optimal control theory. Traditional solutions to the LQT problem 3,4 are offline and are defined in a noncausal manner backward in time. Therefore, they require complete knowledge of the 300