This paper presents an online learning algorithm based on integral reinforcement learning (IRL) to design an output-feedback (OPFB) H ∞ tracking controller for partially unknown linear continuous-time systems. Although reinforcement learning techniques have been successfully applied to find optimal state-feedback controllers, in most control applications, it is not practical to measure the full system states. Therefore, it is desired to design OPFB controllers.To this end, a general bounded L 2 -gain tracking problem with a discounted performance function is used for the OPFB H ∞ tracking. A tracking game algebraic Riccati equation is then developed that gives a Nash equilibrium solution to the associated min-max optimization problem. An IRL algorithm is then developed to solve the game algebraic Riccati equation online without requiring complete knowledge of the system dynamics. The proposed IRL-based algorithm solves an IRL Bellman equation in each iteration online in real time to evaluate an OPFB policy and updates the OPFB gain using the information given by the evaluated policy. An adaptive observer is used to provide the knowledge of the full states for the IRL Bellman equation during learning. However, the observer is not needed after the learning process is finished. A simulation example is provided to verify the convergence of the proposed algorithm to a suboptimal OPFB solution and the performance of the proposed method. KEYWORDSbounded L 2 -gain, H ∞ controller, optimal control, output feedback, reinforcement learning (RL) INTRODUCTIONDesign of output-feedback (OPFB) controllers, particularly static OPFB, 1 is of paramount importance for many applications, such as aircraft control, 2 optimal process control, 3 and elsewhere, since full-state measurement is not usually possible in practical systems. Moreover, the use of OPFB for some tracking control problems allows flexibility and simplicity of implementation.Optimal tracking control problems seek to find a controller that not only stabilizes the tracking error dynamics but also optimizes its transient performance. This is achieved by minimizing a predefined performance index. For linear systems accompanied by a quadratic performance index, the optimal tracking problem is called the linear quadratic tracking (LQT) problem, which is an important problem in the field of optimal control theory. Traditional solutions to the LQT problem 3,4 are offline and are defined in a noncausal manner backward in time. Therefore, they require complete knowledge of the 300

In this paper, we first address adverse effects of cyber-physical attacks on distributed synchronization of multiagent systems, by providing conditions under which an attacker can destabilize the underlying network, as well as another set of conditions under which local neighborhood tracking errors of intact agents converge to zero. Based on this analysis, we propose a Kullback-Liebler divergence based criterion in view of which each agent detects its neighbors' misbehavior and, consequently, forms a self-belief about the trustworthiness of the information it receives. Agents continuously update their selfbeliefs and communicate them with their neighbors to inform them of the significance of their outgoing information. Moreover, if the self-belief of an agent is low, it forms trust on its neighbors. Agents incorporate their neighbors' self-beliefs and their own trust values on their control protocols to slow down and mitigate attacks. We show that using the proposed resilient approach, an agent discards the information it receives from a neighbor only if its neighbor is compromised, and not solely based on the discrepancy among neighbors' information, which might be caused by legitimate changes, and not attacks. The proposed approach is guaranteed to work under mild connectivity assumptions.

An autonomous and resilient controller is proposed for leader-follower multiagent systems under uncertainties and cyber-physical attacks. The leader is assumed nonautonomous with a nonzero control input, which allows changing the team behavior or mission in response to the environmental changes. A resilient learning-based control protocol is presented to find optimal solutions to the synchronization problem in the presence of attacks and system dynamic uncertainties. An observer-based distributed H∞ controller is first designed to prevent propagating the effects of attacks on sensors and actuators throughout the network, as well as to attenuate the effect of these attacks on the compromised agent itself. Nonhomogeneous game algebraic Riccati equations are derived to solve the H∞ optimal synchronization problem and off-policy reinforcement learning (RL) is utilized to learn their solution without requiring any knowledge of the agent's dynamics. A trust-confidence-based distributed control protocol is then proposed to mitigate attacks that hijack the entire node and attacks on communication links. A confidence value is defined for each agent based solely on its local evidence. The proposed resilient RL algorithm employs the confidence value of each agent to indicate the trustworthiness of its own information and broadcast it to its neighbors to put weights on the data they receive from it during and after learning. If the confidence value of an agent is low, it employs a trust mechanism to identify compromised agents and remove the data it receives from them from the learning process. The simulation results are provided to show the effectiveness of the proposed approach.

No abstract