Adaptive Quadruped Balance Control for Dynamic Environments Using Maximum-Entropy Reinforcement Learning

Abstract

External disturbance poses the primary threat to robot balance in dynamic environments. This paper provides a learning-based control architecture for quadrupedal self-balancing, which is adaptable to multiple unpredictable scenes of external continuous disturbance. Different from conventional methods which construct analytical models which explicitly reason the balancing process, our work utilized reinforcement learning and artificial neural network to avoid incomprehensible mathematical modeling. The control policy is composed of a neural network and a Tanh Gaussian policy, which implicitly establishes the fuzzy mapping from proprioceptive signals to action commands. During the training process, the maximum-entropy method (soft actor-critic algorithm) is employed to endow the policy with powerful exploration and generalization ability. The trained policy is validated in both simulations and realistic experiments with a customized quadruped robot. The results demonstrate that the policy can be easily transferred to the real world without elaborate configurations. Moreover, although this policy is trained in merely one specific vibration condition, it demonstrates robustness under conditions that were never encountered during training.

Design an Intelligent Balanced Control of Quadruped Legs Based on Adaptive Neuro-Fuzzy Inference System (ANFIS)

“…Shi et al [2] studied the walking gait of a four-legged animal with a bionic structure capable of performing smooth movement in all directions, inverse kinematic solution using the Denavit-Hartenberg (D-H) method was explored moving the legs of a quadruped robot. Sun et al [3] studied a learning-based control architecture for quadrupedal self-balancing by adapting to some unexpected scenes from continuous external interference. Şen et al [4] simulated a 3-DoF linear leg model with the PIλDµ position control system, a control designed by selecting various fraction order parameters as a comparison with the classic PID control.…”

A Needs Learning Algorithm Applied to Stable Gait Generation of Quadruped Robot

“…Designing controllers manually requires constructing a large number of formulae and performing complex matrix transformations, as well as extensive derivation of formulae and tedious manual adjustments during the design process. When implemented on a physical robot, these methods also need to address random noise and data transmission delays due to hardware problems [ 5 ].…”

“…Lee et al [ 6 ] trained a controller for legged locomotion on challenging terrain via RL in simulation and showed its robustness under real-world conditions, which has never been encountered in simulation training. Sun et al [ 5 ] proposed a convenient and adaptable method to construct a self-balancing controller for a quadruped robot. The method used RL and ANN for policy design, eschewing the construction of kinematic equations, simplifying the design process and enhancing the adaptability of the control strategy.…”