QL-BT: Enhancing behaviour tree design and implementation with Q-learning

Abstract

This is the unspecified version of the paper.This version of the publication may differ from the final published version. Abstract-Artificial intelligence has become an increasingly important aspect of computer game technology, as designers attempt to deliver engaging experiences for players by creating characters with behavioural realism to match advances in graphics and physics. Recently, behaviour trees have come to the forefront of games AI technology, providing a more intuitive approach than previous techniques such as hierarchical state machines, which often required complex data structures producing poorly structured code when scaled up. The design and creation of behaviour trees, however, requires experience and effort. This research introduces Q-learning behaviour trees (QL-BT), a method for the application of reinforcement learning to behaviour tree design. The technique facilitates AI designers' use of behaviour trees by assisting them in identifying the most appropriate moment to execute each branch of AI logic, as well as providing an implementation that can be used to debug, analyse and optimize early behaviour tree prototypes. Initial experiments demonstrate that behaviour trees produced by the QL-BT algorithm effectively integrate RL, automate tree design, and are human-readable. Permanent repository link

Procedural Behavior Model using Behavior Tree in Virtual Reality Applications

“…An example of applying computational operations to behavior tree is a case where a genetic algorithm is used to optimize the components of the behavior tree [8]. There is also an example of studies that allow NPCs or agents to learn on their own using various techniques of reinforcement learning [9].…”

Learning Behavior Trees for Autonomous Agents with Hybrid Constraints Evolution

“…To remedy the disadvantages, in both behavior learning perspectives, there are some attempts to generate behavior models represented as BTs from observation [7,25] or experience [11,12,14,26,27] automatically. In this paper, we are focusing on generate BTs through experiential learning, especially evolving BTs.…”

Synthesis of correct-by-construction behavior trees

“…The approach works well for the kind of applications where the task is to reach a given state and the objective function can be easily derived. In [18] the authors combine BTs with Q-learning, proposing an automated tree design based on reinforcement learning techniques. However, these approaches are applicable where the defined task is to satisfy a single proposition (e.g.…”