Learning Advanced Locomotion for Quadrupedal Robots: A Distributed Multi-Agent Reinforcement Learning Framework with Riemannian Motion Policies

Abstract

Recent advancements in quadrupedal robotics have explored the motor potential of these machines beyond simple walking, enabling highly dynamic skills such as jumping, backflips, and even bipedal locomotion. While reinforcement learning has demonstrated excellent performance in this domain, it often relies on complex reward function tuning and prolonged training times, and the interpretability is not satisfactory. Riemannian motion policies, a reactive control method, excel in handling highly dynamic systems but are generally limited to fully actuated systems, making their application to underactuated quadrupedal robots challenging. To address these limitations, we propose a novel framework that treats each leg of a quadrupedal robot as an intelligent agent and employs multi-agent reinforcement learning to coordinate the motion of all four legs. This decomposition satisfies the conditions for utilizing Riemannian motion policies and eliminates the need for complex reward functions, simplifying the learning process for high-level motion modalities. Our simulation experiments demonstrate that the proposed method enables quadrupedal robots to learn stable locomotion using three, two, or even a single leg, offering advantages in training speed, success rate, and stability compared to traditional approaches, and better interpretability. This research explores the possibility of developing more efficient and adaptable control policies for quadrupedal robots.