Evolving Gaits for Physical Robots with the HyperNEAT Generative Encoding: The Benefits of Simulation

Abstract

Abstract.Creating gaits for physical robots is a longstanding and open challenge. Recently, the HyperNEAT generative encoding was shown to automatically discover a variety of gait regularities, producing fast, coordinated gaits, but only for simulated robots. A follow-up study found that HyperNEAT did not produce impressive gaits when they were evolved directly on a physical robot. A simpler encoding hand-tuned to produce regular gaits was tried on the same robot, and outperformed HyperNEAT, but these gaits were first evolved in simulation before being transferred to the robot. In this paper, we tested the hypothesis that the beneficial properties of HyperNEAT would outperform the simpler encoding if HyperNEAT gaits are first evolved in simulation before being transferred to reality. That hypothesis was confirmed, resulting in the fastest gaits yet observed for this robot, including those produced by nine different algorithms from three previous papers describing gaitgenerating techniques for this robot. This result is important because it confirms that the early promise shown by generative encodings, specifically HyperNEAT, are not limited to simulation, but work on challenging real-world engineering challenges such as evolving gaits for real robots.

Single-unit pattern generators for quadruped locomotion

“…The sine wave architecture, detailed in figure 6, is inspired by Clune et al [8], which first demonstrated HyperNEAT effectively applied to quadruped locomotion. In fact, a later test of this approach on a real quadruped robot yielded the fastest gait yet demonstrated by any optimization method for that model [20] (though of course the robot in this paper has a different morphology). The period of the sine wave is set to one second, which matches the period of the SUPGs (though recall that the actual period realized by the SUPG can vary because of the trigger).…”

“…Popular approaches include neuroevolution (i.e. evolving neural networks) and other evolutionary algorithms, which have achieved some success in approaching the challenge of creating controllers for legged robots [4,8,17,19,20,35,36,38]. Evolutionary algorithms offer the increased flexibility of being possible to re-run for different body morphologies, alleviating the burden of creating a new controller by hand for each new body morphology.…”

“…One popular approach is to enable oscillation by evolving central pattern generators (CPGs) based on continuous time recurrent neural networks (CTRNNs) [3,4,35], which are inspired by mechanisms in the nervous systems of many animals [9,28]. Another approach is to integrate oscillation more directly by inputting a sine wave (or other oscillatory pattern) directly into the artificial neural network (ANN) [8,20,38]. Still another is to compute leg motion as a function of the positions of other legs [36].…”

Investigating Biological Assumptions through Radical Reimplementation

“…They have facilitated an online picture-breeding service [91], an online three-dimensional shape-breeding service [21], an interactive dance evolution program [32], an interactive music evolution program [44], a computer game based on evolved particle weapons [42], formed the foundation for a practical method of evolving complex neural networks [98,24], and have been applied to evolving players for board games like checkers and Go [35,34], virtual creatures in artificial life simulations [3,103], and controllers for both simulated and real robots [54,20]. Interestingly, CPPN-represented shapes evolved through the web service Endless Forms [21] can be realized physically through a threedimensional printer.…”

Intelligent Prediction of Ship Maneuvering

“…These methods are intensively studied and implemented in different fields of science, including robotics (Haasdijk et al, 2010;Lee et al, 2013), automation processes (Kenneth et al, 2005), multi-agent systems (Nowak et al, 2008), designing and diagnostic (Larkin et al, 2006) and many others. Neuroevolutionary algorithms are successful methods for optimizing neural networks topologies, especially in dynamic continuous reinforcement learning tasks.…”