RMA: Rapid Motor Adaptation for Legged Robots

Abstract

Fig. 1: We demonstrate the performance of RMA on several challenging environments. The robot is successfully able to walk on sand, mud, hiking trails, tall grass and dirt pile without a single failure in all our trials. The robot was successful in 70% of the trials when walking down stairs along a hiking trail, and succeeded in 80% of the trials when walking across a cement pile and a pile of pebbles. The robot achieves this high success rate despite never having seen unstable or sinking ground, obstructive vegetation or stairs during training. All deployment results are with the same policy without any simulation calibration, or real-world fine-tuning. Videos at https://ashish-kmr.github.io/rma-legged-robots/ Abstract-Successful real-world deployment of legged robots would require them to adapt in real-time to unseen scenarios like changing terrains, changing payloads, wear and tear. This paper presents Rapid Motor Adaptation (RMA) algorithm to solve this problem of real-time online adaptation in quadruped robots. RMA consists of two components: a base policy and an adaptation module. The combination of these components enables the robot to adapt to novel situations in fractions of a second. RMA is trained completely in simulation without using any domain knowledge like reference trajectories or predefined foot trajectory generators and is deployed on the A1 robot without any fine-tuning. We train RMA on a varied terrain generator using bioenergetics-inspired rewards and deploy it on a variety of difficult terrains including rocky, slippery, deformable surfaces in environments with grass, long vegetation, concrete, pebbles, stairs, sand, etc. RMA shows state-of-the-art performance across diverse real-world as well as simulation experiments. Video results at https://ashish-kmr.github.io/rma-legged-robots/.

Concurrent Training of a Control Policy and a State Estimator for Dynamic and Robust Legged Locomotion

“…Hwangbo et al [1] trained control policies for the ANYmal robot [2] for robust and high-speed locomotion while keeping the balance under large disturbances. In the later works [3] and [4], RL-trained policies made a quadrupedal robot traverse over various challenging terrains such as slippery ground, vegetation, and rocky terrain. They trained an encoder which compresses environmental information and enabled effective environmentaware locomotion.…”

“…The existing control approaches for quadrupedal locomotion rely on accurately estimated state input [1], [3], [4], [7], [10]- [13]. However, we observed that existing This work was supported by Samsung Research Funding & Incubation Center for Future Technology at Samsung Electronics under Project Number SRFC-IT2002-02.…”

Sim-to-Lab-to-Real: Safe Reinforcement Learning with Shielding and Generalization Guarantees

“…Reinforcement Learning (RL) techniques have been increasingly popular in training autonomous robots to perform complex tasks such as traversing uneven outdoor terrains [1] and navigating through cluttered indoor environments [2]. Through interactions with environments and feedback in the form of reward functions, robots learn to reach target locations relying on onboard sensing (e.g., RGB-D cameras).…”

“…However, previous Sim-to-Real techniques do not explicitly address safety of the robots. Usually it is worth compromising the performance (e.g., success rate and time needed for reaching the target) to allow better safety of the system (e.g., rate of bumping into Email addresses: kaichieh@princeton.edu (Kai-Chieh Hsu), allen.ren@princeton.edu (Allen Z. Ren) 1 Equal contributions in alphabetical order 2 Equal contributions in advising Figure 1: Overview of the Sim-to-Lab-to-Real framework. Top: During Sim stage, we train robot policies in a wide variety of environments and conditions.…”

“…This standard pipeline opens up an opportunity that autonomous systems can further improve performance and safety in the Lab stage. Our intuition is that (1) in simulation where environments can be easily randomized and data is easily collected, the robot can be trained in a wide range of indoor environments and conditions; (2) after that, the robot needs to fine-tune in more specific environments in this stage before being deployed in similar environments in the real world, such as office spaces or home living rooms; (3) this training stage can also certify the system before Real deployment, especially if the training can provide guarantees on its performance and safety in the real world. Through such extensive training and validation, we can deploy the system confidently in the real environments.…”