Emergence of Memory-Driven Command Neurons in Evolved Artificial Agents

Abstract

Using evolutionary simulations, we develop autonomous agents controlled by artificial neural networks (ANNs). In simple lifelike tasks of foraging and navigation, high performance levels are attained by agents equipped with fully recurrent ANN controllers. In a set of experiments sharing the same behavioral task but differing in the sensory input available to the agents, we find a common structure of a command neuron switching the dynamics of the network between radically different behavioral modes. When sensory position information is available, the command neuron reflects a map of the environment, acting as a location-dependent cell sensitive to the location and orientation of the agent. When such information is unavailable, the command neuron's activity is based on a spontaneously evolving short-term memory mechanism, which underlies its apparent place-sensitive activity. A two-parameter stochastic model for this memory mechanism is proposed. We show that the parameter values emerging from the evolutionary simulations are near optimal; evolution takes advantage of seemingly harmful features of the environment to maximize the agent's foraging efficiency. The accessibility of evolved ANNs for a detailed inspection, together with the resemblance of some of the results to known findings from neurobiology, places evolved ANNs as an excellent candidate model for the study of structure and function relationship in complex nervous systems.

A robot behavior-learning experiment using Particle Swarm Optimization for training a neural-based animat

“…Our animat model is very similar to that proposed in [13]. The animat behavior is defined by a fully recurrent neural network with 5 sensory inputs, and a total of N fullyconnected neurons among which 4 are motor outputs, as illustrated on figure 1.…”

“…The fifth sensory input is a "smell" binary input indicating if food (rather than poison) is present on the animat's current position, but its value is random when there is nothing on the current cell. As exposed in [13], the idea behind this last "smell" input is that, because the "presence sensor" does not discriminate between food and poison, the eating decision has to be made by fusion of 2 inputs: presence of something on current place, simultaneously with food smell. The 4 motor outputs independently suggest left-rotation, right-rotation, forward-move, and eating.…”

Neural Processing of Counting in Evolved Spiking and McCulloch-Pitts Agents

“…The EAA environment is similar to that of [2]. The agents live in a discrete 2D grid "world" surrounded by walls (Figure 1).…”

“…This study extends upon an evolutionary study of foraging [2], involving an Evolved Autonomous Agent (EAA) living in a 2D grid arena containing food and poison items. In that study, the agent, possessing limited sensory inputs, had to consume as many food items as possible, while avoiding poison items.…”

“…The agent's eating action consisted of standing still on food for one time-step, with its mouth open. Aharonov et al [2] concentrated on the short-term memory mechanism underlying the foraging and navigation behaviors that evolved in this non-stationary environment. Our study aims at finding out whether we can replace the rather simple stimulus-response eating action with a memory-dependent delayed action, in which the agent has to remain still on a food item for an exact constant number of steps before it can consume it.…”