Learning Controller Gains on Bipedal Walking Robots via User Preferences

Abstract

Experimental demonstration of complex robotic behaviors relies heavily on finding the correct controller gains. This painstaking process is often completed by a domain expert, requiring deep knowledge of the relationship between parameter values and the resulting behavior of the system. Even when such knowledge is possessed, it can take significant effort to navigate the nonintuitive landscape of possible parameter combinations. In this work, we explore the extent to which preference-based learning can be used to optimize controller gains online by repeatedly querying the user for their preferences. This general methodology is applied to two variants of control Lyapunov function based nonlinear controllers framed as quadratic programs, which have nice theoretic properties but are challenging to realize in practice. These controllers are successfully demonstrated both on the planar underactuated biped, AMBER, and on the 3D underactuated biped, Cassie. We experimentally evaluate the performance of the learned controllers and show that the proposed method is repeatably able to learn gains that yield stable and robust locomotion.

Disturbance Observers for Robust Safety-Critical Control With Control Barrier Functions

“…Then, under-approximating the size of the uncertainty may yield safety degradation, while over-approximation may in- 13) and (18). duce conservative behavior that is not captured by Theorem 2.…”

“…Example 2. Consider the system in Example 1 and (18). If the uncertainty is under-approximated (p < p), (10) is not satisfied and Theorem 2 cannot establish safety guarantees.…”

“…Corollary 1. Consider (5) with (12) and (18). Assume that there exist δ, δ : H → R ≥0 such that for any h * ∈ H:…”

“…Data-driven methods account for uncertainties in a similar fashion where a sufficient condition for the safety is found using properties of uncertainties [14]- [16]. Learning can also be utilized to estimate the term σ in an episodic fashion [17]-this has been deployed successfully on robotic systems [18]. In the case of stochastic estimation techniques, such as Gaussian processes, probabilistic safety guarantees are obtained using chance constraints with the standard deviation of the process used as an upper confidence bound [19], [20].…”

Data-driven Step-to-step Dynamics based Adaptive Control for Robust and Versatile Underactuated Bipedal Robotic Walking

“…In attempts to improve robotic walking performance and robustness in a data-efficient way, combinations of modelbased and data-driven approaches have been studied. For example, [15], [16] uses data to optimize existing controller parameters. Data-driven reduced order models are considered in [17], [18], where a reduced-order model of the hybrid dynamics or the step-to-step (S2S) dynamics for a specific robot is obtained from simulation data.…”