Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotion

Abstract

Locomotion has seen dramatic progress for walking or running across challenging terrains. However, robotic quadrupeds are still far behind their biological counterparts, such as dogs, which display a variety of agile skills and can use the legs beyond locomotion to perform several basic manipulation tasks like interacting with objects and climbing. In this paper, we take a step towards bridging this gap by training quadruped robots not only to walk but also to use the front legs to climb walls, press buttons, and perform object interaction in the real world. To handle this challenging optimization, we decouple the skill learning broadly into locomotion, which involves anything that involves movement whether via walking or climbing a wall, and manipulation, which involves using one leg to interact while balancing on the other three legs. These skills are trained in simulation using curriculum and transferred to the real world using our proposed sim2real variant that builds upon recent locomotion success. Finally, we combine these skills into a robust longterm plan by learning a behavior tree that encodes a highlevel task hierarchy from one clean expert demonstration. We evaluate our method in both simulation and real-world showing successful executions of both short as well as long-range tasks and how robustness helps confront external perturbations. Videos at https://robot-skills.github.io

Learning agile soccer skills for a bipedal robot with deep reinforcement learning

“…In addition, there is a growing interest in whole-body control, that is, tasks in which the whole body is used in flexible ways to interact with the environment. Examples include getting up from the ground (55) and manipulation of objects with legs (23,56). Recently, RL has been applied to learn simple soccer skills, including goalkeeping (21), ball manipulation (18,19), and shooting (20).…”

“…In particular, deep reinforcement learning (deep RL) has proven capable of solving complex motor control problems for both simulated characters (7)(8)(9)(10)(11) and physical robots. High-quality quadrupedal legged robots have become widely available and have been used to demonstrate behaviors ranging from robust (12,13) and agile (14,15) locomotion to fall recovery (16); climbing (17); basic soccer skills such as dribbling (18,19), shooting (20), intercepting (21), or catching (22) a ball; and simple manipulation with legs (23). On the other hand, much less work has been dedicated to the control of humanoids and bipeds, which impose additional challenges around stability, robot safety, number of degrees of freedom, and availability of suitable hardware.…”

Bridging Locomotion and Manipulation Using Reconfigurable Robotic Limbs via Reinforcement Learning

“…However, due to the topological difference between the quadrupedal base and the manipulator attached, skills cannot be shared between the upper and lower limbs [ 18 ]. Another way of achieving loco-manipulation is to use some or all robotic legs to manipulate objects [ 2 , 19 , 20 , 21 , 22 ]. Recent work presented a goalkeeper quadruped that manipulates a soccer ball using its legs [ 21 ].…”

“…Recent work presented a goalkeeper quadruped that manipulates a soccer ball using its legs [ 21 ]. Locomotion and manipulation skills are trained separately in [ 22 ] and integrated using a behavior tree to achieve long-horizon loco-manipulation tasks such as climbing a wall and pressing a button. These studies successfully learned locomotion and manipulation skills on a single robot, with the robotic limbs functioning as legs and fingers.…”