Development of swarm behavior in artificial learning agents that adapt to different foraging environments

Abstract

Collective behavior, and swarm formation in particular, has been studied from several perspectives within a large variety of fields, ranging from biology to physics. In this work, we apply Projective Simulation to model each individual as an artificial learning agent that interacts with its neighbors and surroundings in order to make decisions and learn from them. Within a reinforcement learning framework, we discuss one-dimensional learning scenarios where agents need to get to food resources to be rewarded. We observe how different types of collective motion emerge depending on the distance the agents need to travel to reach the resources. For instance, strongly aligned swarms emerge when the food source is placed far away from the region where agents are situated initially. In addition, we study the properties of the individual trajectories that occur within the different types of emergent collective dynamics. Agents trained to find distant resources exhibit individual trajectories that are in most cases best fit by composite correlated random walks with features that resemble Lévy walks. This composite motion emerges from the collective behavior developed under the specific foraging selection pressures. On the other hand, agents trained to reach nearby resources predominantly exhibit Brownian trajectories.

Collective Evolution Learning Model for Vision-Based Collective Motion with Collision Avoidance

“…In addition, by approximating the obtained VCMCA model using explainable machine-learning-based models we show that small rule-based models are not able to capture the complexity of the VCMCA model and it is requiring several hundreds of rule combinations to capture only 93.3% of the VCMCA model’s ability while considered out of the scope of the average person or expert [60]. While the proposed model is not explainable on the single decision level, the overall behavior well agrees with known behaviors such as the ones described by [42].…”

“…Similarly, the authors of [41], assume the focal agent is accurately aware of the location and velocities of the neighboring agents while solving a multi-objective task. Nonetheless, they were not able to obtain a collective behavior by learning a policy at the individual level as obtained by Durve et al Furthermore, Lopez-Incera et al [42] studied the collective behavior of artificial learning agents driven by reinforcement learning action-making process and with abstract sensing mechanism, that arises as they attempt to survive in foraging environments. They design multiple foraging scenarios in one-dimensional worlds in which the resources are either near or far from the region where agents are initialized and show the emergence of CM [42].…”

“…Nonetheless, they were not able to obtain a collective behavior by learning a policy at the individual level as obtained by Durve et al Furthermore, Lopez-Incera et al [42] studied the collective behavior of artificial learning agents driven by reinforcement learning action-making process and with abstract sensing mechanism, that arises as they attempt to survive in foraging environments. They design multiple foraging scenarios in one-dimensional worlds in which the resources are either near or far from the region where agents are initialized and show the emergence of CM [42]. While the authors took into consideration a sensing-based approach they investigated a single objective behaviour in an extremely simplistic scenario.…”

Honeybee communication during collective defence is shaped by predation

“…Honeybees offer an interesting opportunity for PS since they exhibit complex behaviours at both the individual and the collective level despite their relatively small brain. In addition, Projective Simulation can be used to model collective behaviour [ 22 , 43 ] by considering ensembles of PS agents that interact with each other. In the present work, this interaction is determined by the olfactory perception of the pheromone that bees release when stinging.…”

“…We show that, with this input, our model accurately predicts the different strategies adopted by each subspecies. Thus, this novel application of Projective Simulation [ 22 ] to a group of agents with a common goal is promising for the study of social insects in general, and of the honeybee defensive behaviour in particular.…”