Neuroevolution of Agents Capable of Reactive and Deliberative Behaviours in Novel and Dynamic Environments

Abstract

Abstract. Both reactive and deliberative qualities are essential for a good action selection mechanism. We present a model that embodies a hybrid of two very different neural network architectures inside an animat: one that controls their high level deliberative behaviours, such as the selection of sub-goals, and one that provides reactive and navigational capabilities. Animats using this model are evolved in novel and dynamic environments, on complex tasks requiring deliberative behaviours: tasks that cannot be solved by reactive mechanisms alone and which would traditionally have their solutions formulated in terms of search-based planning. Significantly, no a priori information is given to the animats, making explicit forward search through state transitions impossible. The complexity of the problem means that animats must first learn to solve sub-goals without receiving any reward. Animats are shown increasingly complex versions of the task, with the results demonstrating, for the first time, incremental neuro-evolutionary learning on such tasks.

Neuroevolution of Feedback Control for Object Manipulation by 3D Agents

“…In our 2015 work, the DN and SM follow the ideas presented in Robinson et al (2007) closely. Together and properly configured, they provide a neural-like encoding of a fixed action policy relating current state (position, local objects and carrying state) to action (preferred movement direction, and a pick-up or put-down action).…”

“…The general problem used in this work is an adaptation of the 3D RC task described in earlier work (Stanton and Channon, 2015), following the same key ideas described and used to various ends in Robinson et al (2007) and Borg and Channon (2011). The innovation in this work is the addition of a requirement for agents to physically manipulate objects in the environment; in our previous work only the body of the agent is physically simulated and all environmental interaction is through a two-dimensional, grid-world abstraction.…”

“…In that work we adapted the shunting model of Grossberg (1988) and Yang and Meng (2000), used for the first time in an a-life context in Robinson et al (2007), to build an evolutionary environment able to evolve control architectures of 3D virtual creatures that exhibit both reactive and deliberative behaviours. However, the problem-solving aspect of the 3D RC task in that work was abstracted from the physicality of the agent's morphology.…”

Incremental Neuroevolution of Reactive and Deliberative 3D Agents

“…For example, using 2D non-articulated agent bodies, early work by Yaeger showed (in a 3D environment) the emergence of complex collective behaviour (Yaeger, 1993); Channon demonstrated the first candidate synthetic open-ended evolutionary system using an agent-based (2D) world (Channon and Damper, 1998);and Robinson et al (2007) evolved agents capable of reactive and deliberative behaviours in novel and dynamic environments.…”

“…The "River Crossing" (RC) task devised by Robinson et al (2007) is used as the baseline reactive-deliberative problem. This task is adapted by the addition of a requirement of physical motor control in 3D, and the complete problem against which agents are tested is hereafter referred to as the 3D River Crossing or 3D RC task.…”