Combining Simulation and Reality in Evolutionary Robotics

Abstract

Evolutionary Robotics (ER) is a promising methodology, intended for the autonomous development of robots, in which their behaviors are obtained as a consequence of the structural coupling between robot and environment. It is essential that there be a great amount of interaction to generate complex behaviors. Thus, nowadays, it is common to use simulation to speed up the learning process; however simulations are achieved from arbitrary off-line designs, rather than from the result of embodied cognitive processes. According to the reality gap problem, controllers evolved in simulation usually do not allow the same behavior to arise once transferred to the real robot. Some preliminary approaches for combining simulation and reality exist in the ER literature; nonetheless, there is no satisfactory solution available. In this work we discuss recent advances in neuroscience as a motivation for the use of environmentally adapted simulations, which can be achieved through the co-evolution of robot behavior and simulator. We present an algorithm in which only the differences between the behavior fitness obtained in reality versus that obtained in simulations are used as feedback for adapting a simulation. The proposed algorithm is experimentally validated by showing the successful development and continuous transference to reality of two complex low-level behaviors with Sony AIBO 1 robots: gait optimization and ball-kicking behavior.

Beyond black-box optimization: a review of selective pressures for evolutionary robotics

“…proposed the back-to-reality algorithm [189,188,187], a similar approach that consists in performing an optimization in simulation, transferring some selected solutions to reality and exploiting the corresponding data to improve the simulation before optimizing in simulation again. These different steps are repeated until the behavioral requirements are met.…”

“…These different steps are repeated until the behavioral requirements are met. The approach has been used for a locomotion task on a quadruped robot [189,188] and on a humanoid robot [187]. Farchy et al propose a similar approach with a choice made by the experimenter on which parameters to focus on for the next optimization [57].…”

The distributed co-evolution of an on-board simulator and controller for swarm robot behaviours

“…Evolving walking gait behaviours, Bongard et al [2] develop the 'estimation-exploration' algorithm which utilises evaluations in reality to capture limb-joint sensor data to adapt an offline off-board simulation of the robot morphology. Zagal et al [25] develop the 'Back To Reality' algorithm, which co-evolves an offline off-board simulation of a quadruped robot and a walking gait controller, by using a single measure of discrepancy between the achieved walking gait in simulation versus reality.…”

“…Zagal et al [25] describe the potential utility of an on-board simulator in terms of an incorporated aspect of an embodied robot controller, drawing analogy to the faculty of dreaming in cognitive neuroscience. This work proposes a different utility; an on-board simulator may aid the aforementioned problems with an online on-board distributed evolutionary approach.…”

A Comparison of Sampling Strategies for Parameter Estimation of a Robot Simulator

“…The approaches taken in Back-to-Reality (BTR) [20,21], estimationexploration (EE) [22,23], and using sequential surrogate optimization (SSO) [24] are largely similar to each other: Two coupled optimization algorithms are run in an interleaved fashion, one to search for solutions to a primary task such as simulated robot locomotion and another to improve the accuracy of the simulator. The product of each run of the primary search (a controller for a simulated robot) is evaluated in reality and then used in subsequent simulator optimizations; and the product of each simulator optimization is used in the following primary search.…”