Agile Maneuvers in Legged Robots: a Predictive Control Approach

Mastalli, Carlos; Merkt, Wolfgang; Xin, Guiyang; Shim, Jae-Hyun; Mistry, Michael; Havoutis, Ioannis; Vijayakumar, Sethu

doi:10.48550/arxiv.2203.07554

Cited by 7 publications

(16 citation statements)

References 74 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Model Predictive Control (MPC) 20 enables a system to make current decisions while considering their impact on the future through predictive modeling, which has shown promise in recent research on legged locomotion. Recently, some complex leg models, such as single rigid-body models 21,22 , central dynamics models 23,24 , and whole-body models 25 , have been used to improve the locomotor skills of legged robots, especially quadruped robots. These MPC-based strategies focus on the unified processing of legged motions and can find reliable motion trajectories in real time, which are robust to external disturbances.…”

Section: Related Workmentioning

confidence: 99%

Learning Agility and Adaptive Legged Locomotion via Curricular Hindsight Reinforcement Learning

Li,

Pang,

Bai

et al. 2024

Preprint

View full text Add to dashboard Cite

Agile and adaptive maneuvers such as fall recovery, high-speed turning, and sprinting in the wild are challenging for legged systems. We propose a Curricular Hindsight Reinforcement Learning (CHRL) that learns an end-to-end tracking controller that achieves powerful agility and adaptation for the legged robot. The two key components are (i) a novel automatic curriculum strategy on task difficulty and (ii) a Hindsight Experience Replay strategy adapted to legged locomotion tasks. We demonstrated successful agile and adaptive locomotion on a real quadruped robot that performed fall recovery autonomously, coherent trotting, sustained outdoor running speeds up to 3.45 m/s, and tuning speeds up to 3.2 rad/s. This system produces adaptive behaviors responding to changing situations and unexpected disturbances on natural terrains like grass and dirt.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning Agility and Adaptive Legged Locomotion via Curricular Hindsight Reinforcement Learning

Li,

Pang,

Bai

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Model Predictive Control (MPC) [19] enables a system to make current decisions while considering their impact on the future through predictive modeling, which has shown promise in recent research on legged locomotion. Recently, some complex leg models, such as single rigid-body models [20,21], central dynamics models [22,23], and whole-body models [24], have been used to improve the locomotor skills of legged robots, especially quadruped robots. These MPC-based strategies focus on the unified processing of legged motions and can find reliable motion trajectories in real time, which are robust to external disturbances.…”

Section: Related Workmentioning

confidence: 99%

Dynamic Fall Recovery Control for Legged Robots via Reinforcement Learning

Li,

Pang,

Bai

et al. 2024

Biomimetics

View full text Add to dashboard Cite

Falling is inevitable for legged robots when deployed in unstructured and unpredictable real-world scenarios, such as uneven terrain in the wild. Therefore, to recover dynamically from a fall without unintended termination of locomotion, the robot must possess the complex motor skills required for recovery maneuvers. However, this is exceptionally challenging for existing methods, since it involves multiple unspecified internal and external contacts. To go beyond the limitation of existing methods, we introduced a novel deep reinforcement learning framework to train a learning-based state estimator and a proprioceptive history policy for dynamic fall recovery under external disturbances. The proposed learning-based framework applies to different fall cases indoors and outdoors. Furthermore, we show that the learned fall recovery policies are hardware-feasible and can be implemented on real robots. The performance of the proposed approach is evaluated with extensive trials using a quadruped robot, which shows good effectiveness in recovering the robot after a fall on flat surfaces and grassland.

show abstract

“…Until recently, most approaches have used simple models, such as single rigid body (3) or inverted pendulum dynamics (4,5), or have ignored the dynamic effects altogether (6). Research has shifted toward more complex formulations, including centroidal (7) or full-body dynamics (8). The resulting trajectories are tracked by a whole-body control (WBC) module, which operates at the control frequency and uses full-body dynamics (9).…”

Section: Introductionmentioning

confidence: 99%

“…Although the problem of sparse gradients still exists, two important advantages can be exploited: First, cost function and constraint gradients can be computed with a small number of samples. Second, poor local optima can be avoided by precomputing footholds (5,8), presegmenting the terrain into steppable areas (7,20), or smoothing out the entire gradient landscape (21). Another advantage of TO is its ability to plan actions ahead and predict future interactions with the environment.…”

Section: Introductionmentioning

confidence: 99%

DTC: Deep Tracking Control

Jenelten,

He,

Farshidian

et al. 2024

Sci. Robot.

View full text Add to dashboard Cite

Legged locomotion is a complex control problem that requires both accuracy and robustness to cope with real-world challenges. Legged systems have traditionally been controlled using trajectory optimization with inverse dynamics. Such hierarchical model-based methods are appealing because of intuitive cost function tuning, accurate planning, generalization, and, most importantly, the insightful understanding gained from more than one decade of extensive research. However, model mismatch and violation of assumptions are common sources of faulty operation. Simulation-based reinforcement learning, on the other hand, results in locomotion policies with unprecedented robustness and recovery skills. Yet, all learning algorithms struggle with sparse rewards emerging from environments where valid footholds are rare, such as gaps or stepping stones. In this work, we propose a hybrid control architecture that combines the advantages of both worlds to simultaneously achieve greater robustness, foot-placement accuracy, and terrain generalization. Our approach uses a model-based planner to roll out a reference motion during training. A deep neural network policy is trained in simulation, aiming to track the optimized footholds. We evaluated the accuracy of our locomotion pipeline on sparse terrains, where pure data-driven methods are prone to fail. Furthermore, we demonstrate superior robustness in the presence of slippery or deformable ground when compared with model-based counterparts. Last, we show that our proposed tracking controller generalizes across different trajectory optimization methods not seen during training. In conclusion, our work unites the predictive capabilities and optimality guarantees of online planning with the inherent robustness attributed to offline learning.

show abstract

Agile Maneuvers in Legged Robots: a Predictive Control Approach

Cited by 7 publications

References 74 publications

Learning Agility and Adaptive Legged Locomotion via Curricular Hindsight Reinforcement Learning

Learning Agility and Adaptive Legged Locomotion via Curricular Hindsight Reinforcement Learning

Dynamic Fall Recovery Control for Legged Robots via Reinforcement Learning

DTC: Deep Tracking Control

Contact Info

Product

Resources

About