A Semi-Markov Decision Model With Inverse Reinforcement Learning for Recognizing the Destination of a Maneuvering Agent in Real Time Strategy Games

Abstract

Recognizing the destination of a maneuvering agent is important to create intelligent AI players in Real Time Strategy (RTS) games. Among different ways of problem formulation, goal recognition can be solved as a model-based planning problem using off-the-shelf planners. However, the common problem in these frameworks is that they usually lack of the modeling of the action duration as in real-world scenarios the agent may take several steps to transit between grids. To solve this problem, a semi-Markov decision model (SMDM), which explicitly models the duration of an action, is proposed in this paper. Besides, most of the current works do not establish a behavioral model of the identified person, and there is almost no work modeling individual behavioral preference, which limits the accuracy of the recognition results. In this paper, the Inverse Reinforcement Learning (IRL) method is adopted in opponent behavior learning for the destination recognition problem. To adapt to the dynamic environment, the Maximum Entropy Inverse Reinforcement Learning (MaxEnt IRL) method is transformed by defining a Fitness index to measure the effect of weight and use the Nelder-Mead polyhedron search to find the optimal weight. In experiments, we build the game scenario in the Unreal Engine 4 environment and collect the moving trajectories from the human players in several different tasks for evaluating the performance of our methods. The results show that the recognizer using IRL can recognize the destination effectively even if the intention changes during the midway, and it performs better than other models in terms of several most frequently-used metrics. INDEX TERMS Real time strategy games, goal recognition, inverse reinforcement learning.

Object Detection Recognition and Robot Grasping Based on Machine Learning: A Survey

“…Robot [147,151,156,184] Computer vision [185][186][187][188][189][190] Game [191][192][193] Autonomous driving [185,186]…”

AI intelligent wayfinding based on Unreal Engine 4 static map

“…The development of artificial intelligence promotes to the development of video games.Video games have five advantages: real simulation, definition of boundary, non-destructive exploration, god standard and fun.With the continuous optimization of the high fidelity of the game engine,the game can provide highly simulated training environment [1].Therefore, the game can be used as the carrier of artificial intelligence algorithm testing, and the effect of artificial intelligence algorithm can also be verified in the game. Excellent artificial intelligence technology can offer more realistic and enjoyable gaming experience to players.AI intelligent path-finding is the core component of the game, and it is also the focus of research on game artificial intelligence [2].The nature of path-finding algorithm is path search that is also called path planning, that is, finding a feasible path between two points.In the game, NPC (Non-Player Character), that is, AI character, realizes movement in the game map according to the planned path based on path-finding algorithm [3].The game has high requirements for real-time responsiveness of AI characters. Solving the path search problem effcetively can increase the authenticity and interest of the game.…”