Learning Character Behaviors Using Agent Modeling in Games

Abstract

Our goal is to provide learning mechanisms to game agents so they are capable of adapting to new behaviors based on the actions of other agents. We introduce a new on-line reinforcement learning (RL) algorithm, ALeRT-AM, that includes an agent-modeling mechanism. We implemented this algorithm in BioWare Corp.’s role-playing game, Neverwinter Nights to evaluate its effectiveness in a real game. Our experiments compare agents who use ALeRT-AM with agents that use the non-agent modeling ALeRT RL algorithm and two other non-RL algorithms. We show that an ALeRT-AM agent is able to rapidly learn a winning strategy against other agents in a combat scenario and to adapt to changes in the environment.

Learning Companion Behaviors Using Reinforcement Learning in Games

“…Techniques such as behavior trees (Isla 2005) and rule based (Spronck et al 2006) methods have been used in games. Recently, RL has been used to enable NPCs to learn behavior strategies for combat scenarios (Cutumisu et al) (Zhao and Szafron 2009). However, there have been no successful attempts to enable companion NPCs to learn more flexible behaviors that are responsive to changes in emotional and physical state.…”

“…Most progress on using RL in games has been on learning high-level strategies rather than behaviors for individual NPCs. However, Cutumisu et al (2008) and Zhao and Szafron (2009) have shown that individual NPCs can learn behaviors using variations of the Sarsa(λ) algorithm called ALeRT, and ALeRT-AM. These algorithms have dynamic learning rates that support the fast changing environments found in video games.…”

Learning From Stories: Using Crowdsourced Narratives to Train Virtual Agents

“…Reinforcement learning has been successfully used to train virtual characters (Zhao and Szafron 2009) and for drama management before (Nelson and Mateas 2005;Roberts et al 2006); however complex design knowledge must first be provided that encode the designers intuitions about how characters behave and scenarios should unfold. The work presented in this paper allows the same knowledge to be automatically acquired from natural language interactions (in this case, from crowdsourced narrative examples) and converted into a form suitable to train virtual characters: a reward function.…”

ScriptEase II: Platform Independent Story Creation Using High-Level Patterns

“…Second, it can be used by both independent game designers and novices who are learning to design games. Third, it can be used by academics and game companies to study the use of AI in computer games (Zhao and Szafron 2009).…”