Evolutionary Robotics, Anticipation and the Reality Gap

Hartland, Cédric; Bredèche, Nicolas

doi:10.1109/robio.2006.340190

Cited by 15 publications

(11 citation statements)

References 8 publications

(8 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The robot can also explicitly build an approximate model of its environment to use it as a reference and then adapt to environment variations. For instance in [21], an anticipation module allowed to build a model of the motor consequences in the simulated environment. If some differences are encountered once in reality between this model and the current environment, a correction module performs online adaptation to improve the behavior and overcome the gap.…”

Section: B Simulation-based Optimizationmentioning

confidence: 99%

The Transferability Approach: Crossing the Reality Gap in Evolutionary Robotics

Koos

Mouret

Doncieux

2013

IEEE Trans. Evol. Computat.

216

184

View full text Add to dashboard Cite

Abstract-The reality gap, that often makes controllers evolved in simulation inefficient once transferred onto the physical robot, remains a critical issue in Evolutionary Robotics (ER). We hypothesize that this gap highlights a conflict between the efficiency of the solutions in simulation and their transferability from simulation to reality: the most efficient solutions in simulation often exploit badly modeled phenomena to achieve high fitness values with unrealistic behaviors. This hypothesis leads to the Transferability approach, a multi-objective formulation of ER in which two main objectives are optimized via a Pareto-based Multi-Objective Evolutionary Algorithm: (1) the fitness and (2) the transferability, estimated by a simulation-to-reality (STR) disparity measure. To evaluate this second objective, a surrogate model of the exact STR disparity is built during the optimization. This Transferability approach has been compared to two realitybased optimization methods, a noise-based approach inspired from Jakobi's minimal simulation methodology and a local search approach. It has been validated on two robotic applications: 1) a navigation task with an e-puck robot; 2) a walking task with an 8-DOF quadrupedal robot. For both experimental set-ups, our approach successfully finds efficient and well-transferable controllers only with about ten experiments on the physical robot.

show abstract

Section: B Simulation-based Optimizationmentioning

confidence: 99%

The Transferability Approach: Crossing the Reality Gap in Evolutionary Robotics

Koos

Mouret

Doncieux

2013

IEEE Trans. Evol. Computat.

216

184

View full text Add to dashboard Cite

show abstract

“…where k indicate the situation index and K is the total number of situations. Moreover, i t,k is an indicator variable describing whether s t is assigned to the k-th situation as defined by (3).…”

Section: ) Situation Definitionmentioning

confidence: 99%

“…Traditional optimization algorithms are difficult to use for the optimizing behavior of UV due to high computational complexity of those methods [3]. To overcome the limitations of the existing optimization algorithms, instead of attempting to obtain an exact solution, heuristic or learning-based algorithms were proposed in previous studies as summarized in Table I.…”

Section: Introductionmentioning

confidence: 99%

A Behavior Optimization Method for Unmanned Combat Aerial Vehicles Using Matrix Factorization

et al. 2020

View full text Add to dashboard Cite

One of the fundamental technologies for unmanned combat aerial vehicles and combat simulators is behavior optimization, which finds a behavior that maximizes the probability of winning a battle. With the advent of military science, combat logs became available, allowing machine learning algorithms to be used for the behavior optimization. Due to implicit attributes such as the experience of an operator that are not explicitly presented in log data, existing methods for behavior optimization have limitations in performance improvement. Furthermore, specific behaviors occur with low frequency, resulting in a dataset with imbalanced and empty values. Therefore, we apply a matrix factorization (MF) method, which is one of latent factor models and known for sophisticated imputation of empty values, to the behavior optimization problem of unmanned combat aerial vehicles. A situation-behavior matrix, whose elements are ratings indicating the optimality of behaviors in situations, is defined to implement the MF based method. Experiments for performance comparison were conducted on combat logs, in which the proposed method yielded satisfactory results. INDEX TERMS behavior optimization, unmanned vehicle, matrix factorization, reinforcement learning, situation-behavior matrix ABBREVIATIONS AM Advantage matrix. FOV Field of view. GA Genetic algorithm. LOS Line of sight. MF Matrix factorization. nDCG Normalized discounted cumulative gain. RL Reinforcement learning. SB Situation-behavior. UV Unmanned vehicle.

show abstract

“…The goal is then to find flexible enough controllers that behave well in simulation, but can adapt online to the gap once transferred onto the real robot. In [10], an anticipation module allows to build a model of the motor consequences in the simulated environment. Then, if some differences are encountered once in reality between this model and the current environment, a correction module performs online adaptation to improve the behavior and overcome the gap.…”

Section: Related Workmentioning

confidence: 99%

Crossing the reality gap in evolutionary robotics by promoting transferable controllers

Koos

Mouret

Doncieux

2010

Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation

View full text Add to dashboard Cite

The reality gap, that often makes controllers evolved in simulation inefficient once transferred onto the real system, remains a critical issue in Evolutionary Robotics (ER); it prevents ER application to real-world problems. We hypothesize that this gap mainly stems from a conflict between the efficiency of the solutions in simulation and their transferability from simulation to reality: best solutions in simulation often rely on bad simulated phenomena (e.g. the most dynamic ones). This hypothesis leads to a multi-objective formulation of ER in which two main objectives are optimized via a Pareto-based Multi-Objective Evolutionary Algorithm: (1) the fitness and (2) the transferability. To evaluate this second objective, a simulation-to-reality disparity value is approximated for each controller. The proposed method is applied to the evolution of walking controllers for a real 8-DOF quadrupedal robot. It successfully finds efficient and well-transferable controllers with only a few experiments in reality.

show abstract

Evolutionary Robotics, Anticipation and the Reality Gap

Cited by 15 publications

References 8 publications

The Transferability Approach: Crossing the Reality Gap in Evolutionary Robotics

The Transferability Approach: Crossing the Reality Gap in Evolutionary Robotics

A Behavior Optimization Method for Unmanned Combat Aerial Vehicles Using Matrix Factorization

Crossing the reality gap in evolutionary robotics by promoting transferable controllers

Contact Info

Product

Resources

About