Approximate Nash Solutions for Multiplayer Mixed-Zero-Sum Game With Reinforcement Learning

H∞ Control for Discrete-Time Multi-Player Systems via Off-Policy Q-Learning

“…Most related results are H ∞ control of multi-agent and multi-player systems [6]- [9]. In [7] agents have their individual dynamics and anti-interference problem has been investigated for continuous-time multi-player systems in [6], [8], [9]. The main difference of multi-player systems from multi-agent systems is all players in multi-player systems are capable of accessing the state of the overall systems, which makes us try to find a different manipulation from multi-agent systems to design H ∞ controllers.…”

“…The main difference of multi-player systems from multi-agent systems is all players in multi-player systems are capable of accessing the state of the overall systems, which makes us try to find a different manipulation from multi-agent systems to design H ∞ controllers. Like [6], [8], [9], the model-free H ∞ controller design will be taken into account for multi-player systems in this paper, while the difference of nature of discrete-time sampling from continues-time processes makes it more complicated to solve H ∞ control problem from the discrete-time system perspective, and multiple players and completely unknown dynamics of players increase this difficulty. Moreover, in view of the advantages of off-policy learning over on-policy learning shown in our previous result [37] wherein the off -policy Q-learning method was proposed for multi-player systems without the consideration of disturbance, developing an off-policy game Q-learning algorithm to solve H ∞ control problem for discrete-time linear multi-player systems using only measured data becomes our target.…”

“…The contributions of this paper are summarized as follows: 1) Unlike the on-policy algorithm [21], [23], [36], [39]- [41] and the off-policy RL method [27], [28] which only consider single-player systems, this paper employs the idea of off-policy Q-learning and extends it to multi-player systems for handling H ∞ control of multi-player systems. Compared with the existing H ∞ control algorithms of multi-agent systems [7] and multi-player systems [6], [8], [9], there are two differences, one is model-free H ∞ controller is designed in this paper for discrete-time dynamics of systems rather than continuous-time dynamics of systems [6]- [9] and the other is that all players in the focused systems cooperatively work hard for achieving their common objective, i.e. minimizing the specific performance and defeating external disturbances, which is not like the way in [6], [8], [9] where players competed each other for optimizing its own performance.…”

Online Dual-Network-Based Adaptive Dynamic Programming for Solving Partially Unknown Multi-Player Non-Zero-Sum Games With Control Constraints

“…T HE theories with respect to differential games have received more and more attentions since it was firstly studied in [1]. With the efforts of worldwide scholars, differential games theories, which are closely linked to our daily life nowadays, have been widely used in economics, sociology and many other domains [2]- [7]. Three indispensable elements, i.e., players, control policies and performance functions, jointly build the footstone of differential games theory.…”

Data‐Driven Approximated Optimal Control of Sulfur Flotation Process

“…Adaptive dynamic programming is an integration of adaptive control, dynamic programming, and reinforcement learning [14,15]. By using an "actor-critic" structure, it can approximate the solution of the Hamilton-Jacobi-Bellman (HJB) equation online without knowing the full knowledge of the system model, or even when the system model is unknown [16,17]. Thus, if an initial admissible control policy is available, an approximated optimal control can be obtained by solving the HJB equation iteratively.…”