Robust Biped Locomotion Using Deep Reinforcement Learning on Top of an Analytical Control Approach

Kasaei, Mohammadreza; Abreu, Miguel; Lau, Nuno; Pereira, Artur; Reis, Luis Paulo

doi:10.48550/arxiv.2104.10592

Cited by 3 publications

(3 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition to learning skills from scratch, reinforcement learning algorithms are also used to optimize existing skills. Kasaei et al [102] used a combination of reinforcement learning and robot dynamics models to generate and optimize the omnidirectional walking skill of robots. This study used the ZMP as the main criterion for robot stability and designed a linear inverted pendulum model considering the motion of the center of mass.…”

Section: Reinforcement Learning Methodsmentioning

confidence: 99%

A survey of research on several problems in the RoboCup3D simulation environment

Gao,

Yi,

Jin

et al. 2024

Auton Agent Multi-Agent Syst

View full text Add to dashboard Cite

In the process of robot research and development, due to the vulnerability of hardware, simulation environment is often used to verify and test algorithms first. RoboCup3D simulation environment is developed based on open dynamic engine, and the humanoid robot NAO is modeled as the main robot, which provides a simulation platform for humanoid robot researchers to study robot movements. At the same time, it is also the official platform of RoboCup 3D events. Under the rules of soccer robot competition, it is helpful for the research of multi-robots, especially multi-humanoid robots' cooperation strategy. This paper summarizes the related research in RoboCup3D simulation environment, and first introduces the basic problems existing in this simulation environment. Secondly, the research of robot motion generation and optimization based on model and nonmodel in simulation environment is introduced respectively. Then, it introduces the related research of cooperation strategy design of multi-humanoid robots under RoboCup3D rules, including positioning, dynamic role assignment, etc. And sort out a typical practical solution to the above problems; Finally, the future development trend of related research in RoboCup3D simulation environment is analyzed.

show abstract

Section: Reinforcement Learning Methodsmentioning

confidence: 99%

A survey of research on several problems in the RoboCup3D simulation environment

Gao,

Yi,

Jin

et al. 2024

Auton Agent Multi-Agent Syst

View full text Add to dashboard Cite

show abstract

“…The reference trajectory of the robot can be used to regulate robot behavior and narrow the exploration space of the RL-based policy. While many prior approaches use dynamics models to generate physically feasible reference trajectories for bipedal locomotion [11,12], using fixed reference trajectories proved to be feasible for quadrupedal locomotion [4] as the control policy is capable of modifying the trajectory to stabilize the robot. In our framework, we use a reference trajectory generator to build trajectory, regardless of the robot's current state.…”

Section: Control Frameworkmentioning

confidence: 99%

“…Reinforcement learning has been employed to control low-speed quadrupedal walking on uneven terrains [4][5][6][7], or high-speed running on flat ground [8,9]. RL is also implemented in bipedal locomotion controllers, resulting in a better performance regarding dynamic model uncertainty [10], fast yet robust maneuvers [11,12], and the precise control of stepping point [13] or gait [14].…”

Section: Introduction 1backgroundmentioning

confidence: 99%

Learning Quadrupedal High-Speed Running on Uneven Terrain

Han,

Zhao

2024

Biomimetics

View full text Add to dashboard Cite

Reinforcement learning (RL)-based controllers have been applied to the high-speed movement of quadruped robots on uneven terrains. The external disturbances increase as the robot moves faster on such terrains, affecting the stability of the robot. Many existing RL-based methods adopt higher control frequencies to respond quickly to the disturbance, which requires a significant computational cost. We propose a control framework that consists of an RL-based control policy updating at a low frequency and a model-based joint controller updating at a high frequency. Unlike previous methods, our policy outputs the control law for each joint, executed by the corresponding high-frequency joint controller to reduce the impact of external disturbances on the robot. We evaluated our method on various simulated terrains with height differences of up to 6 cm. We achieved a running motion of 1.8 m/s in the simulation using the Unitree A1 quadruped. The RL-based control policy updates at 50 Hz with a latency of 20 ms, while the model-based joint controller runs at 1000 Hz. The experimental results show that the proposed framework can overcome the latency caused by low-frequency updates, making it applicable for real-robot deployment.

show abstract

A survey of research on several problems in the RoboCup3D simulation environment

Gao

Jin

et al. 2023

Preprint

View full text Add to dashboard Cite

In the process of robot research and development, due to the vulnerability of hardware, simulation environment is often used to verify and test algorithms first. RoboCup3D simulation environment is developed based on open dynamic engine, and the humanoid robot NAO is modeled as the main robot, which provides a simulation platform for humanoid robot researchers to study robot movements. At the same time, it is also the official platform of RoboCup 3D events. Under the rules of soccer robot competition, it is helpful for the research of multi-robots, especially multi-humanoid robots' cooperation strategy. This paper summarizes the related research in RoboCup3D simulation environment, and first introduces the basic problems existing in this simulation environment. Secondly, the research of robot motion generation and optimization based on model and non-model in simulation environment is introduced respectively. Then, it introduces the related research of cooperation strategy design of multi-humanoid robots under RoboCup3D rules, including positioning, dynamic role assignment, etc. And sort out a typical practical solution to the above problems; Finally, the future development trend of related research in RoboCup3D simulation environment is analyzed.

show abstract

Robust Biped Locomotion Using Deep Reinforcement Learning on Top of an Analytical Control Approach

Cited by 3 publications

References 44 publications

A survey of research on several problems in the RoboCup3D simulation environment

A survey of research on several problems in the RoboCup3D simulation environment

Learning Quadrupedal High-Speed Running on Uneven Terrain

A survey of research on several problems in the RoboCup3D simulation environment

Contact Info

Product

Resources

About