Empirical assessment and comparison of neuro-evolutionary methods for the automatic off-line design of robot swarms

Abstract

Neuro-evolution is an appealing approach to generating collective behaviors for robot swarms. In its typical application, known as off-line automatic design, the neural networks controlling the robots are optimized in simulation. It is understood that the so-called reality gap, the unavoidable differences between simulation and reality, typically causes neural network to be less effective on real robots than what is predicted by simulation. In this paper, we present an empirical study on the extent to which the reality gap impacts the most popular and advanced neuro-evolutionary methods for the off-line design of robot swarms. The results show that the neural networks produced by the methods under analysis performed well in simulation, but not in real-robot experiments. Further, the ranking that could be observed in simulation between the methods eventually disappeared. We find compelling evidence that real-robot experiments are needed to reliably assess the performance of neuro-evolutionary methods and that the robustness to the reality gap is the main issue to be addressed to advance the application of neuro-evolution to robot swarms.

Balancing Collective Exploration and Exploitation in Multi-Agent and Multi-Robot Systems: A Review

“…Currently, a large proportion of experimental results available tend to be limited in size, scope, and robustness. This highlights the need for more extensive experimentation involving larger systems operating in more complex environments (Hasselmann et al, 2021).…”

Social learning in swarm robotics

“…small robots for monitoring, pest destruction or pollinating) or biomedical applications (e.g. drug delivery and bio-sensing), for the mere reason that real-world physics is extremely challenging to model, even in a simple controlled setting [ 10 ]. As recently noted by several authors [ 49 – 52 ], dense robot swarms, where many physical collisions occur, can be considered from a statistical physics perspective as an active matter.…”

“…The learned strategy is then deployed in an operational situation without any further learning, as global monitoring and evaluation are no longer available. Despite the various problems of using controlled environments during learning [ 10 ], this form of prior-to-deployment learning has been successfully applied to several problems, from training builder robots for construction [ 7 ] to autonomous high sea boats for surveillance [ 17 ] and outdoor UAVs reconstructing a communication network after a natural disaster [ 18 ].…”

AutoMoDe-Mate: Automatic off-line design of spatially-organizing behaviors for robot swarms

“…Unfortunately, although neuro-evolution is a versatile approach, it is prone to suffer from bootstrapping and deception [53]. Also, it has been observed that the performance of neuro-evolutionary methods is substantially affected by the so-called reality gap [11,22,23,53,61,62]-that is, the difference between simulations models and reality.…”

“…Neuro-evolution and modular design are both appealing approaches to the automatic generation of robot control software. A larger body of literature belongs to the former [6] and, only recently, studies started to shed light on how popular methods of the two approaches compare to each other [23].…”