Preference-Based Learning for User-Guided HZD Gait Generation on Bipedal Walking Robots

Abstract

This paper presents a framework that unifies control theory and machine learning in the setting of bipedal locomotion. Traditionally, gaits are generated through trajectory optimization methods and then realized experimentallya process that often requires extensive tuning due to differences between the models and hardware. In this work, the process of gait realization via hybrid zero dynamics (HZD) based optimization problems is formally combined with preferencebased learning to systematically realize dynamically stable walking. Importantly, this learning approach does not require a carefully constructed reward function, but instead utilizes human pairwise preferences. The power of the proposed approach is demonstrated through two experiments on a planar biped AMBER-3M: the first with rigid point feet, and the second with induced model uncertainty through the addition of springs where the added compliance was not accounted for in the gait generation or in the controller. In both experiments, the framework achieves stable, robust, efficient, and natural walking in fewer than 50 iterations with no reliance on a simulation environment. These results demonstrate a promising step in the unification of control theory and learning.

Hands-free Telelocomotion of a Wheeled Humanoid toward Dynamic Mobile Manipulation via Teleoperation

“…As such, both pilots requested that the wheel position tracking gains of the stabilization controller be reduced significantly. These findings suggest that our stabilization controller with a human in the loop may benefit from preference based control designs [21].…”

Intelligent design for quadruped robot on a dynamic flexible surface with an active spine

“…The input layer maps the bending sensor signals into the Reservoir Computing space, while the reservoir layer is composed of a randomly connected network of neurons that transforms the input into a high-dimensional feature space. 14,15 The Reservoir Computing layer then feeds the transformed input into a linear readout layer to estimate the system's response, the benefit for Reservoir Computing is that it able to analysis chaotic system which suitable for soft robotics dynamic modeling. 13,16 The only trainable part of reservoir computing is the readout, which is ridge node, can be trained by state of training and y[t], where the state of training is the activation of the reservoir trigger by x[t], 10 as shown in Figure 1.…”

Learning Controller Gains on Bipedal Walking Robots via User Preferences

“…We refer to this setting as "preference-based learning". In this work, we utilize a more recent preference-based learning algorithm, LineCoSpar [23] with the addition of ordinal labels inspired from [24], which maintains the posterior only over a subset of the entire actions space to increase computation tractability -more details can be found in [14]. The resulting learning framework iteratively applies Thompson sampling to navigate a high-dimensional Bayesian landscape of user preferences.…”