Combining Simulations and Real-Robot Experiments for Bayesian Optimization of Bipedal Gait Stabilization

Abstract

Walking controllers often require parametrization which must be tuned according to some cost function. To estimate these parameters, simulations can be performed which are cheap but do not fully represent reality. Real-robot experiments, on the other hand, are more expensive and lead to hardware wear-off. In this paper, we propose an approach for combining simulations and real experiments to learn gait stabilization parameters. We use a Bayesian optimization method which selects the most informative points in parameter space to evaluate based on the entropy of the cost function to optimize. Experiments with the igus Humanoid Open Platform demonstrate the effectiveness of our approach. * Both authors contributed equally.

NimbRo Robots Winning RoboCup 2018 Humanoid AdultSize Soccer Competitions

“…Using Bayesian optimization, we rely not only on real-world experiments but also on simulated experiments to gain useful information, without wearing off the hardware of the robot. This approach has already been successfully applied to the igus Humanoid Open Platform [25] and the NimbRo-OP2X [2].…”

“…This trade-off depends on a kernel function k and the parametrization of the underlying Gaussian Process (GP). The latter encodes problem-specific values like signal noise and can be measured by a series of initial experiments [25]. The proposed kernel, on the other hand, is composed of two components, where the first term k sim encodes simulation performance and the second term k functions as an error-term resembling the difference between simulation and the real-world performance:…”

NimbRo-OP2X: Adult-Sized Open-Source 3D Printed Humanoid Robot

“…On the other hand, simulations can be performed without any cost, risk, and-most importantly-fast. In our recent work [11], simulation and real-robot experiments were combined for gait optimisation of smaller robots. In this paper, we extend this work to our new larger robot, with more complex gait sequences, to achieve a self-tuned gait that is able to withstand strong impacts.…”

“…To minimise hardware wear-off, this optimisation does not only take place in the real world, but highly exploits information gained through the included Gazebo simulator. This approach has been previously applied by Rodriguez et al [11] and is now utilised on the NimbRo-OP2X robot.…”

“…7. Although it might seem that the simulations only had a small influence on the result, they were of utmost importance in early iterations [11]. This is underlined especially by the fact, that the robot did not fall a single time during realworld evaluations, which proves that the GP model is able to utilise simulation data to rule out inferior parameters.…”