Optimal game theoretic solution of the pursuit‐evasion intercept problem using on‐policy reinforcement learning

Abstract

This article presents a rigorous formulation for the pursuit‐evasion (PE) game when velocity constraints are imposed on agents of the game or players. The game is formulated as an infinite‐horizon problem using a non‐quadratic functional, then sufficient conditions are derived to prove capture in a finite‐time. A novel tracking Hamilton–Jacobi–Isaacs (HJI) equation associated with the non‐quadratic value function is employed, which is solved for Nash equilibrium velocity policies for each agent with arbitrary nonlinear dynamics. In contrast to the existing remedies for proof of capture in PE game, the proposed method does not assume players are moving with their maximum velocities and considers the velocity constraints a priori. Attaining the optimal actions requires the solution of HJI equations online and in real‐time. We overcome this problem by presenting the on‐policy iteration of integral reinforcement learning (IRL) technique. The persistence of excitation for IRL to work is satisfied inherently until capture occurs, at which time the game ends. Furthermore, a nonlinear backstepping control method is proposed to track desired optimal velocity trajectories for players with generalized Newtonian dynamics. Simulation results are provided to show the validity of the proposed methods.

Solution for Pursuit-Evasion Game of Agents by Adaptive Dynamic Programming

“…We can obtain the controls of pursuer and evader which should be adopted in the next interval as (30):…”

“…However, the system information about both sides of the game must be obtained completely. Kartal et al [30] used the synchronous tuning algorithm in the pursuit-evasion game of the first-order system to obtain the capture conditions of agents in the game and reached the Nash equilibrium. Zhang et al [31] and Li et al [32] determined the scheme's feasibility in distributed systems.…”

Optimal-Damage-Effectiveness Cooperative-Control Strategy for the Pursuit–Evasion Problem with Multiple Guided Missiles

“… Due to the limitation of the driving force, the input of the movement mode of the pursuer and the evader is limited to the constants and . The input of the movement mode of the pursuer also satisfies a certain proportion relationship [ 23 , 24 ]. Because of the above assumption, the pursuer is slower than the evader, i.e., The location of the line of defense is determined, as are the number of pursuers and evaders.…”

Target defense against a sequentially arriving cooperative intruder team

“…[7][8][9][10] Likewise, perimeter-defense problems are another variant of PEGs wherein the defender team is tasked to capture the intruders before the latter breach the target perimeter. Hamilton-Jacobi-Bellman-Isaacs equation is one of the conventional tools to address perimeter-defense problems, 11,12 however, this is not suitable for team games or the type of sequential arrival games considered in this work. A perimeter defense problem in a planar conical environment is studied 13 recently where two algorithms were presented.…”