Data-Driven Nonzero-Sum Game for Discrete-Time Systems Using Off-Policy Reinforcement Learning

Abstract

In this paper, we develop a data-driven algorithm to learn the Nash equilibrium solution for a two-player non-zero-sum (NZS) game with completely unknown linear discrete-time dynamics based on offpolicy reinforcement learning (RL). This algorithm solves the coupled algebraic Riccati equations (CARE) forward in time in a model-free manner by using the online measured data. We first derive the CARE for solving the two-player NZS game. Then, model-free off-policy RL is developed to obviate the requirement of complete knowledge of system dynamics. Besides, on-and off-policy RL algorithms are compared in terms of the robustness against the probing noise. Finally, a simulation example is presented to show the efficacy of the presented approach. INDEX TERMS Coupled algebraic Riccati equations, off-policy reinforcement learning, nonzero-sum game.

An efficient model‐free adaptive optimal control of continuous‐time nonlinear non‐zero‐sum games based on integral reinforcement learning with exploration

“…Various RL algorithms [15][16][17][18] exhibit diverse computational complexities, leading to varying demands for computational power. Therefore, it becomes imperative to study the method that can reduce the computational power requirements of algorithms without compromising their performance.…”

“…The third type is the actor-critic (AC) structure, which is also the most widely used. For example, [16][17][18] respectively solved the NZS problem of different controlled systems. There is also a synchronous RL method [28,29] based on the AC structure, which can continuously and simultaneously adjust the weights of actor NN and critic NN.…”

Off‐policy integral reinforcement learning‐based optimal tracking control for a class of nonzero‐sum game systems with unknown dynamics

“…Many practical application scenarios can be modeled as a multi‐input system which is controlled by multiple controllers 1,2 . From the perspective of game theory, 3‐5 the study of optimal control problems for multi‐control input systems has become a hotspot in control theory research 6‐10 . Based on the different roles and tasks of each control input, optimal control problem of multi‐input system can be divided into: fully cooperative games (FC), 11 zero‐sum (ZS) games, 12 and nonzero‐sum (NZS) games 13 .…”

“…1,2 From the perspective of game theory, [3][4][5] the study of optimal control problems for multi-control input systems has become a hotspot in control theory research. [6][7][8][9][10] Based on the different roles and tasks of each control input, optimal control problem of multi-input system can be divided into: fully cooperative games (FC), 11 zero-sum (ZS) games, 12 and nonzero-sum (NZS) games. 13 In fact, FC games can be regarded as a special case of the NZS games.…”

Data-Driven Adaptive Dynamic Programming for Optimal Control of Continuous-Time Multicontroller Systems With Unknown Dynamics

“…Based on the roles and tasks of inputs, the optimal control of multiple control input systems can be studied from three perspectives: zero-sum game (ZS), non-zero-sum (NZS) game and fully cooperative (FC) game [40]. For zero-sum games [41], [42] [43], [44] and non-zero-sum games [45], [46] [47], [48] [49], [50] [51], [52], scholars have developed many ADP methods. How, there are few ADP studies on fully cooperative games [53], [54].…”