Legged robots pose one of the greatest challenges in robotics. Dynamic and agile maneuvers of animals cannot be imitated by existing methods that are crafted by humans. A compelling alternative is reinforcement learning, which requires minimal craftsmanship and promotes the natural evolution of a control policy. However, so far, reinforcement learning research for legged robots is mainly limited to simulation, and only few and comparably simple examples have been deployed on real systems. The primary reason is that training with real robots, particularly with dynamically balancing systems, is complicated and expensive. In the present work, we report a new method for training a neural network policy in simulation and transferring it to a state-of-the-art legged system, thereby we leverage fast, automated, and cost-effective data generation schemes. The approach is applied to the ANYmal robot, a sophisticated medium-dog-sized quadrupedal system. Using policies trained in simulation, the quadrupedal machine achieves locomotion skills that go beyond what had been achieved with prior methods: ANYmal is capable of precisely and energy-efficiently following high-level body velocity commands, running faster than ever before, and recovering from falling even in complex configurations.

This paper introduces ANYmal, a quadrupedal robot that features outstanding mobility and dynamic motion capability. Thanks to novel, compliant joint modules with integrated electronics, the 30 kg, 0.5 m tall robotic dog is torque controllable and very robust against impulsive loads during running or jumping. The presented machine was designed with a focus on outdoor suitability, simple maintenance, and user-friendly handling to enable future operation in real world scenarios. Performance tests with the joint actuators indicated a torque control bandwidth of more than 70 Hz, high disturbance rejection capability, as well as impact robustness when moving with maximal velocity. It is demonstrated in a series of experiments that ANYmal can execute walking gaits, dynamically trot at moderate speed, and is able to perform special maneuvers to stand up or crawl very steep stairs. Detailed measurements unveil that even full-speed running requires less than 280 W, resulting in an autonomy of more than 2 h.

This paper provides a system overview about ANYmal, a quadrupedal robot developed for operation in harsh environments. The 30 kg, 0.5 m tall robotic dog was built in a modular way for simple maintenance and user-friendly handling, while focusing on high mobility and dynamic motion capability. The system is tightly sealed to reach IP67 standard and protected to survive falls. Rotating lidar sensors in the front and back are used for localization and terrain mapping and compact force sensors in the feet provide accurate measurements about the contact situations. The variable payload, such as a modular pan-tilt head with a variety of inspection sensors, can be exchanged depending on the application. Thanks to novel, compliant joint modules with integrated electronics, ANYmal is precisely torque controllable and very robust against impulsive loads during running or jumping. In a series experiments we demonstrate that ANYmal can execute various climbing maneuvers, walking gaits, as well as a dynamic trot and jump. As special feature, the joints can be fully rotated to switch between X-and O-type kinematic configurations. Detailed measurements unveil a low energy consumption of 280 W during locomotion, which results in an autonomy of more than 2 h.

This paper addresses the problem of legged locomotion in non-flat terrain. As legged robots such as quadrupeds are to be deployed in terrains with geometries which are difficult to model and predict, the need arises to equip them with the capability to generalize well to unforeseen situations. In this work, we propose a novel technique for training neuralnetwork policies for terrain-aware locomotion, which combines state-of-the-art methods for model-based motion planning and reinforcement learning. Our approach is centered on formulating Markov decision processes using the evaluation of dynamic feasibility criteria in place of physical simulation. We thus employ policy-gradient methods to independently train policies which respectively plan and execute foothold and base motions in 3D environments using both proprioceptive and exteroceptive measurements. We apply our method within a challenging suite of simulated terrain scenarios which contain features such as narrow bridges, gaps and stepping-stones, and train policies which succeed in locomoting effectively in all cases. * These authors contributed equally.

We show that learned policies can be applied to solve legged locomotion control tasks with extensive flight phases, such as those encountered in space exploration. Using an offthe-shelf deep reinforcement learning algorithm, we trained a neural network to control a jumping quadruped robot while solely using its limbs for attitude control. We present tasks of increasing complexity leading to a combination of threedimensional (re-)orientation and landing locomotion behaviors of a quadruped robot traversing simulated low-gravity celestial bodies. We show that our approach easily generalizes across these tasks and successfully trains policies for each case. Using sim-toreal transfer, we deploy trained policies in the real world on the SpaceBok robot placed on an experimental testbed designed for two-dimensional micro-gravity experiments. The experimental results demonstrate that repetitive, controlled jumping and landing with natural agility is possible.Index Terms-Legged Robots; Deep Learning in Robotics and Automation; Space Robotics and Automation * We choose to use the gravity of Ceres, −0.27m/s 2 ≈ 0.03g [22]. Ceres is the smallest dwarf planet of the solar system. The low gravity environment makes it an interesting target for jumping locomotion.† https://youtu.be/KQhlZa42fe4