Cat-Like Jumping and Landing of Legged Robots in Low Gravity Using Deep Reinforcement Learning

Abstract

We show that learned policies can be applied to solve legged locomotion control tasks with extensive flight phases, such as those encountered in space exploration. Using an offthe-shelf deep reinforcement learning algorithm, we trained a neural network to control a jumping quadruped robot while solely using its limbs for attitude control. We present tasks of increasing complexity leading to a combination of threedimensional (re-)orientation and landing locomotion behaviors of a quadruped robot traversing simulated low-gravity celestial bodies. We show that our approach easily generalizes across these tasks and successfully trains policies for each case. Using sim-toreal transfer, we deploy trained policies in the real world on the SpaceBok robot placed on an experimental testbed designed for two-dimensional micro-gravity experiments. The experimental results demonstrate that repetitive, controlled jumping and landing with natural agility is possible.Index Terms-Legged Robots; Deep Learning in Robotics and Automation; Space Robotics and Automation * We choose to use the gravity of Ceres, −0.27m/s 2 ≈ 0.03g [22]. Ceres is the smallest dwarf planet of the solar system. The low gravity environment makes it an interesting target for jumping locomotion.† https://youtu.be/KQhlZa42fe4

An Optimal Motion Planning Framework for Quadruped Jumping

“…There are some works that use RL to deal with quadruped robot jumping. They are used to address the (re)orientation problem of the robot's 3D posture during the jumping flight phase in the case of low gravity (e.g., moon) [7], or compensating for the error of the jumping trajectory caused by disturbance [8], and training the robot to have cat-like action to ensure the landing phase's safety. However, while these systems have the advantage of transferring the policy to the robot's onboard computer after training and computing the necessary behavior from policy in a short time.…”

“…However, while these systems have the advantage of transferring the policy to the robot's onboard computer after training and computing the necessary behavior from policy in a short time. [7], [9] However, extensive data collecting is required in the early stages. Meanwhile, it does not develop a motion planning policy to conduct many complex jumping, such as doing a left-flip and a back-flip at the same time, nor does it consider the problem of optimal energy consumption to select the best trajectory from plausible options.…”

Bingham Policy Parameterization for 3D Rotations in Reinforcement Learning

“…Deep reinforcement learning (RL) is now actively used in many areas, including playing games [25,33], robot manipulation [24,16], and legged robotics [13,29]. The leading (general-purpose) algorithms within the continuous control RL community are either deterministic, such as DDPG [23] and TD3 [5], or stochastic, such as SAC [11] and PPO [31].…”

Real-time Optimal Landing Control of the MIT Mini Cheetah

“…Several other approaches such as reinforcement learning or model-predictive control (MPC) present interesting landing behaviors also demonstrated on hardware. [9] demonstrated planar landing and airborne orientation control on the SpaceBok quadruped in a "low-gravity" environment, but it is unclear how easily the algorithm could be applied for real-world, 3D conditions with more dramatic impacts and inertial effects. While planar landing and jumping was also demonstrated in [10] and [11], touchdown was made without considering optimal touchdown positions or timings, and were from relatively low heights with little pitch or roll of the body.…”