Towards a Deep Reinforcement Learning Approach for Tower Line Wars

Abstract

Abstract. There have been numerous breakthroughs with reinforcement learning in the recent years, perhaps most notably on Deep Reinforcement Learning successfully playing and winning relatively advanced computer games. There is undoubtedly an anticipation that Deep Reinforcement Learning will play a major role when the first AI masters the complicated game plays needed to beat a professional Real-Time Strategy game player. For this to be possible, there needs to be a game environment that targets and fosters AI research, and specifically Deep Reinforcement Learning. Some game environments already exist, however, these are either overly simplistic such as Atari 2600 or complex such as Starcraft II from Blizzard Entertainment. We propose a game environment in between Atari 2600 and Starcraft II, particularly targeting Deep Reinforcement Learning algorithm research. The environment is a variant of Tower Line Wars from Warcraft III, Blizzard Entertainment. Further, as a proof of concept that the environment can harbor Deep Reinforcement algorithms, we propose and apply a Deep Q-Reinforcement architecture. The architecture simplifies the state space so that it is applicable to Q-learning, and in turn improves performance compared to current state-of-the-art methods. Our experiments show that the proposed architecture can learn to play the environment well, and score 33% better than standard Deep Q-learning -which in turn proves the usefulness of the game environment.

Reinforcement Learning for Extended Reality: Designing Self-Play Scenarios

“…Initially, self-play agents were trained to play boardgames (such as chess and go, among others) [20] but it has now been successfully extended from the classic and simpler Atari 2600 video games [25] to more complex first-person shooters (Doom [26], Battlefield [27], Quake [28]), Role Playing games 1 (Warcraft [29]), Real-Time Strategic Games (Starcraft [30][31][32]) and more recently Multiplayer Online Battle Arena (Dota 2 [33]). For a more comprehensive review see the work by Justesen et al [34].…”

The Dreaming Variational Autoencoder for Reinforcement Learning Environments

“…The DVAE algorithm was tested on two game environments. The first environment is Deep Line Wars [1], a simplified Real-Time Strategy game. We introduce Deep Maze, a flexible environment with a wide range of challenges suited for reinforcement learning research.…”

“…This extends the capabilities of Deep Maze to support nearly all possible scenario combination in the realm of maze solving. 1 State Representation RL agents depend on sensory input to evaluate and predict the best action at current timestep. Preprocessing of data is essential so that agents can extract features from the input.…”

“…The Deep Line Wars environment was first introduced in [1]. Deep Line Wars is a real-time strategy environment that makes an extensive state-space reduction to enable swift research in reinforcement learning for RTS games.…”

“…The DVAE algorithm works well in more complex environments, such as the Deep Line Wars game environment [1]. Here, we expand the DVAE algorithm with LSTM to improve the interpretation of animations, illustrated Figure 1.…”