Evolving Multimodal Networks for Multitask Games

Abstract

Abstract-Intelligent opponent behavior helps make video games interesting to human players. Evolutionary computation can discover such behavior, especially when the game consists of a single task. However, multitask domains, in which separate tasks within the domain each have their own dynamics and objectives, can be challenging for evolution. This paper proposes two methods for meeting this challenge by evolving neural networks: 1) Multitask Learning provides a network with distinct outputs per task, thus evolving a separate policy for each task, and 2) Mode Mutation provides a means to evolve new output modes, as well as a way to select which mode to use at each moment. Multitask Learning assumes agents know which task they are currently facing; if such information is available and accurate, this approach works very well, as demonstrated in the Front/Back Ramming game of this paper. In contrast, Mode Mutation discovers an appropriate task division on its own, which may in some cases be even more powerful than a human-specified task division, as shown in the Predator/Prey game of this paper. These results demonstrate the importance of both Multitask Learning and Mode Mutation for learning intelligent behavior in complex games.

Evolving multimodal behavior with modular neural networks in Ms. Pac-Man

“…A similar approach is Mode Mutation [21], which introduces complete modules rather than individual neurons; a single Mode Mutation adds enough output neurons to define a new policy, plus an additional neuron to arbitrate between modules. The behavior-defining neurons are called policy neurons, and the one arbitration neuron per module is called a preference neuron.…”

“…Different versions of Module Mutation have been evaluated in prior research [21]. MM(P) initializes new modules with lateral inputs from a previous module.…”

“…Each module represents a different policy the agent can use, and special preference neurons are used to arbitrate between available modules. The number of modules can be fixed, or evolved using Module Mutation (also called Mode Mutation [21]). A new version of this structural mutation operator-called Module Mutation Duplicate because new modules copy the behavior of a previous module-is developed and evaluated in this paper.…”

Evolution of sustained foraging in three-dimensional environments with physics

“…Simulations in 3D physical environments are much easier to set up compared to their hardware counterpart, they can be executed many times faster than real time, and allow the same experiment to be reproduced perfectly (see, e.g., [48]). Simulated environments with physics are also extensively used outside the field of robotics, such as the simulation of physical phenomena in computer graphics [49][50][51][52][53], and gaming [54], to name but a few. In biology, environments with a full simulation of physics can be used to study the evolution and usage of physical and behavioral traits that are specifically adapted to overcome the constraints dictated by the simulated laws of physics, and can help better understand how similar traits evolved and are used by real organisms [55][56][57][58].…”

Short-Term Memory Mechanisms in the Goal-Directed Behavior of the Neural Network Agents

“…Одним из наиболее широко используемых подходов к разработке адаптивных контроллеров в многоцелевых средах на данный момент является нейроэволюция [10][11][12].…”