Learning hybrid locomotion skills—Learn to exploit residual actions and modulate model-based gait control

Abstract

This work has developed a hybrid framework that combines machine learning and control approaches for legged robots to achieve new capabilities of balancing against external perturbations. The framework embeds a kernel which is a model-based, full parametric closed-loop and analytical controller as the gait pattern generator. On top of that, a neural network with symmetric partial data augmentation learns to automatically adjust the parameters for the gait kernel, and also generate compensatory actions for all joints, thus significantly augmenting the stability under unexpected perturbations. Seven Neural Network policies with different configurations were optimized to validate the effectiveness and the combined use of the modulation of the kernel parameters and the compensation for the arms and legs using residual actions. The results validated that modulating kernel parameters alongside the residual actions have improved the stability significantly. Furthermore, The performance of the proposed framework was evaluated across a set of challenging simulated scenarios, and demonstrated considerable improvements compared to the baseline in recovering from large external forces (up to 118%). Besides, regarding measurement noise and model inaccuracies, the robustness of the proposed framework has been assessed through simulations, which demonstrated the robustness in the presence of these uncertainties. Furthermore, the trained policies were validated across a set of unseen scenarios and showed the generalization to dynamic walking.

Neural dynamics of robust legged robots

“…Through bio-mechanical analysis, we found that the most robust agents agents, trained via the Terrain-Disturbance policy, exhibit rapid RH hip actuation when responding to a surprise lateral disturbance. To understand why this agile response arises, we look at disturbance-based legged locomotion studies of insects ( Jindrich and Full, 2002 ; Revzen et al, 2013 ), humans Hof et al (2010) , and robots ( Kasaei et al, 2023 ). In ( Hof et al, 2010 ), it is concluded that humans recover from lateral perturbations by taking a wider next step and also shifting their center of pressure through ankle adjustment.…”

“…After training, we perform physical perturbations trials, where an external lateral force is applied to the robot at its center of mass for 80 ms, which is a similar duration as found in other works ( Hof et al, 2010 ; Kasaei et al, 2023 ). Unless otherwise specified, we apply a perturbation of 3.5 times body weight at the moment when the LF and RH foot hit the ground.…”

“…In this work, we draw inspiration from ( Meyes et al, 2019 ; Towlson and Barabási, 2020 ), where ablations are extended from single neurons to pairs and triplets, as well as ( Jonas and Kording, 2017 ), which suggests that lesioning studies could be more meaningful if we could simultaneously control the system at a given moment. Our method of controlling our agent is by applying precisely-timed surprise lateral perturbations, similar to those studied in animals ( Karayannidou et al, 2009 ; Hof et al, 2010 ), and robots ( Kasaei et al, 2023 ). However, our focus is not just on the bio-mechanical response, but on the neural activity that ultimately drives this stabilizing behavior.…”

“…Our method of controlling our agent is by applying preciselytimed surprise lateral perturbations, similar to those studied in animals (Karayannidou et al, 2009;Hof et al, 2010), and robots (Kasaei et al, 2023). However, our focus is not just on the biomechanical response, but on the neural activity that ultimately drives this stabilizing behavior.…”