Learning to Play General Video-Games via an Object Embedding Network

Abstract

Deep reinforcement learning (DRL) has proven to be an effective tool for creating general video-game AI. However most current DRL video-game agents learn end-to-end from the video-output of the game, which is superfluous for many applications and creates a number of additional problems. More importantly, directly working on pixel-based raw video data is substantially distinct from what a human player does. In this paper, we present a novel method which enables DRL agents to learn directly from object information. This is obtained via use of an object embedding network (OEN) that compresses a set of object feature vectors of different lengths into a single fixedlength unified feature vector representing the current game-state and fulfills the DRL simultaneously. We evaluate our OEN-based DRL agent by comparing to several state-of-the-art approaches on a selection of games from the GVG-AI Competition. Experimental results suggest that our object-based DRL agent yields performance comparable to that of those approaches used in our comparative study.

Learning-Based Video Game Development in MLP@UoM: An Overview

“…From experimental results, we do observe that there are notable difference in performance between pixel-level and objectbased representations. It is evident from the experimental results that our object-based learnable agent is capable of autonomous learning in all five games and leads to better generalisation performance or robustness immune to noise and the background variation in raw video data (see [24] for detailed results).…”

“…Abundant evidence suggests that human players work on objects in a video game via perceptual organisation of coherent pixels rather than treating all the pixels in a frame independently. To this end, we have developed object-based reinforcement learning techniques [24] to create "human-like" learnable agents.…”

A Framework for End-to-End Learning on Semantic Tree-Structured Data

“…without history based policy) Proximal Policy Optimisation algorithm [32], using our JSON network model to encode the environment state into a single latent vector, which we branch off into separate policy and value heads using a single linear transformation. Since the game-state consists of multiple object-level descriptions, for encoding this final list we use the architecture used in [33] (which is a variant of the architecture of [1]), which is applied via path-specific function mapping. Additionally, since various object attributes are cosmetic, or otherwise irrelevant, we ignore these via path-specific function mappings.…”

“…For our baseline we use the object-based method of [33] using the same object-level features, but adapted for the PPO algorithm. This uses the same root level architecture, but with hand-defined object-level feature vectors.…”

Exchangeable Input Representations for Reinforcement Learning

“…Tasks with varying numbers of objects are often solved with ad-hoc approaches such as input zero-padding. These methods can often lead to training inefficiencies [15]. We show that the proposed attention mechanism can accept varying numbers of input objects without ad-hoc approximation.…”