Evolving a Behavioral Repertoire for a Walking Robot

Abstract

Numerous algorithms have been proposed to allow legged robots to learn to walk. However, most of these algorithms are devised to learn walking in a straight line, which is not sufficient to accomplish any real-world mission. Here we introduce the Transferability-based Behavioral Repertoire Evolution algorithm (TBR-Evolution), a novel evolutionary algorithm that simultaneously discovers several hundreds of simple walking controllers, one for each possible direction. By taking advantage of solutions that are usually discarded by evolutionary processes, TBR-Evolution is substantially faster than independently evolving each controller. Our technique relies on two methods: (1) novelty search with local competition, which searches for both high-performing and diverse solutions, and (2) the transferability approach, which combines simulations and real tests to evolve controllers for a physical robot. We evaluate this new technique on a hexapod robot. Results show that with only a few dozen short experiments performed on the robot, the algorithm learns a repertoire of controllers that allows the robot to reach every point in its reachable space. Overall, TBR-Evolution introduced a new kind of learning algorithm that simultaneously optimizes all the achievable behaviors of a robot.

Evolutionary Policy Transfer and Search Methods for Boosting Behavior Quality: RoboCup Keep-Away Case Study

“…Recent work in Evolutionary Robotics (ER) (Doncieux et al, 2015) has provided increasing empirical evidence that maintaining diversity in genotypes (robot controller encodings) and phenotypes (robot behaviors) improves the quality (task performance) of evolved behaviors in a range of tasks (Mouret and Doncieux, 2012;Cully et al, 2015;Cully and Mouret, 2016;Gomes et al, 2016).…”

“…Recent work in Evolutionary Robotics (ER) (Doncieux et al, 2015) has provided increasing empirical evidence that maintaining diversity in phenotypes (robot behaviors) improves the quality (task performance) of evolved behaviors (Mouret and Doncieux, 2012;Cully et al, 2015;Cully and Mouret, 2016;Gomes et al, 2016). Specifically, replacing objective search with the search for behavioral diversity in controller evolution (Moriguchi and Honiden, 2010;Mouret and Doncieux, 2012;Lehman et al, 2013;Gomes et al, 2015) has been demonstrated to boost the quality of evolved behaviors across a range of simulated (Lehman and Stanley, 2011a;Mouret and Doncieux, 2012;Gomes et al, 2016) and physical (Cully et al, 2015;Cully and Mouret, 2016) ER tasks.…”

“…Specifically, replacing objective search with the search for behavioral diversity in controller evolution (Moriguchi and Honiden, 2010;Mouret and Doncieux, 2012;Lehman et al, 2013;Gomes et al, 2015) has been demonstrated to boost the quality of evolved behaviors across a range of simulated (Lehman and Stanley, 2011a;Mouret and Doncieux, 2012;Gomes et al, 2016) and physical (Cully et al, 2015;Cully and Mouret, 2016) ER tasks.…”

“…In controller design in the field of ER, there has been an increasing research and empirical data indicating that non-objective evolutionary search, such as novelty search (Lehman and Stanley, 2011a) and other behavioral diversity maintenance approaches (Mouret and Doncieux, 2012), out-perform objective-based search in various evolutionary robotic control tasks defined by complex, high dimensional, and deceptive fitness landscapes (Cully et al, 2015;Cully and Mouret, 2016;Gomes et al, 2016). However, current empirical data indicate that for controller evolution to solve complex collective behavior tasks, then neither objective nor nonobjective-based search performs well (evolves high-quality behaviors).…”

Incremental Neuroevolution of Reactive and Deliberative 3D Agents

“…The seminal work by Sims (1994) remains an exemplar to the present day. Subsequent research has made incremental steps from this point, including demonstrating realistic co-adapted behaviours using just general purpose neurons (Miconi and Channon, 2006), making use of a human-specified syllabus of reactive locomotion-based tasks (Lessin et al, 2013) and using Novely Search (Lehman and Stanley, 2008) to evolve a range of gaits for a fixed morphology robot (Cully and Mouret, 2015), but continues to focus primarily on locomotion alone, leaving more complex behaviours aside.…”