Intelligent controller for passivity-based biped robot using deep Q network

Wu, Yao; Yao, Daojin; Xiao, Xiaohui; Guo, Zhao

doi:10.3233/jifs-172180

Cited by 12 publications

(6 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…30 Besides, adaptive model-free controllers using RL were proposed to stabilize PDW-based robots, improving robustness to disturbance and time varying environments, while maintain PDW’s merits of human-like motion and high energy efficiency. 31,32 So this paper expanded out previous work to improve the PDW-based robot’s versatility by stabilizing its chaotic behaviors.…”

Section: Introductionmentioning

confidence: 94%

A hybrid chaotic controller integrating hip stiffness modulation and reinforcement learning-based torque control to stabilize passive dynamic walking

Qiao

Yao

2022

Proceedings of the Institution of Mechanical Engineers, Part C:

Self Cite

View full text Add to dashboard Cite

Implementation of bipedal stable walking has attracted a lot of interest. Passive dynamic walking (PDW) is one promising manner to generate natural bipedal walking with high energy efficiency. However, how to improve stability and versatility of PDW-based robot against disturbance and time varying environments is still a big challenge in robotics. Chaos and bifurcations are intrinsic features of biped dynamic walking, which are important reasons for the failure of PDW. Thus a hybrid chaotic controller to stabilize chaos and bifurcations of PDW was proposed in this paper. At first, the dynamics model of compliant biped robot was set up, and routine to bipedal chaotic motions was found through parameter study. Then one hybrid chaotic controller integrating hip stiffness modulation and hip impulse torque control was developed, where hip impulse torque control based on reinforcement learning was trained to get state variables close to fixed point of PDW, and then hip stiffness modulation was conducted based on Ott-Grebogi-Yorke method to stabilize unstable motions of fixed point. Simulation results showed that period-1 stable walking could be gained for biped robots from chaotic motions, against original value disturbance, force disturbance and in time varying environments. The proposed hybrid chaotic controller could be used to stabilize bipedal chaotic motions and make the passivity-based robot become robust and versatile to disturbed and time changing environments.

show abstract

Section: Introductionmentioning

confidence: 94%

A hybrid chaotic controller integrating hip stiffness modulation and reinforcement learning-based torque control to stabilize passive dynamic walking

Qiao

Yao

2022

Proceedings of the Institution of Mechanical Engineers, Part C:

Self Cite

View full text Add to dashboard Cite

show abstract

“…Humanoid robots are high-dimensional non-smooth systems with many physical constraints. Nowadays, an increasing number of humanoid robot control algorithms are being developed adopting DRL in research on, for example, robot balance [3][4][5], the dynamic performance after the training of a legged robot [6] and the passive dynamic walking robot [7]. Vuga et al [8] used motion capture with a model-free reinforcement learning algorithm to reduce differences in humanoid robots learning human actions.…”

Section: Biped Robot Controlled By Drlmentioning

confidence: 99%

A Disturbance Rejection Control Method Based on Deep Reinforcement Learning for a Biped Robot

et al. 2021

View full text Add to dashboard Cite

The disturbance rejection performance of a biped robot when walking has long been a focus of roboticists in their attempts to improve robots. There are many traditional stabilizing control methods, such as modifying foot placements and the target zero moment point (ZMP), e.g., in model ZMP control. The disturbance rejection control method in the forward direction of the biped robot is an important technology, whether it comes from the inertia generated by walking or from external forces. The first step in solving the instability of the humanoid robot is to add the ability to dynamically adjust posture when the robot is standing still. The control method based on the model ZMP control is among the main methods of disturbance rejection for biped robots. We use the state-of-the-art deep-reinforcement-learning algorithm combined with model ZMP control in simulating the balance experiment of the cart–table model and the disturbance rejection experiment of the ASIMO humanoid robot standing still. Results show that our proposed method effectively reduces the probability of falling when the biped robot is subjected to an external force in the x-direction.

show abstract

“…Figure 4 shows the basic structure of the AUV hunting path planning implemented by the DRL method. The AUV uses DRL and a dual-stream Q-network to avoid obstacles [37][38]. By setting a negative reward for collision, we pursue a strategy to maximize the total reward.…”

Section: Path Planning Of Huntingmentioning

confidence: 99%

“…A target speed of 2 grids/second. The AUVs complete the hunting at the target marching point (32,38,37). Under the same conditions, the proposed algorithm uses five AUVs to successfully close the target at (28,25,22), and the hunting distance is shortened by 48%, as shown in Figure 9(b).…”

Section: Figure 8 the Structure Diagram Of Bio-inspired Neural Networkmentioning

confidence: 99%

Hunting Algorithm for Multi-AUV Based on Dynamic Prediction of Target Trajectory in 3D Underwater Environment

Cao

2020

IEEE Access

View full text Add to dashboard Cite

In the research of multi-robot systems, multi-AUV (multiple autonomous underwater vehicles) cooperative target hunting is a hot issue. In order to improve the target hunting efficiency of multi-AUV, a multi-AUV hunting algorithm based on dynamic prediction for the trajectory of the moving target is proposed in this paper. Firstly, with moving of the target, sample points are updated dynamically to predict the possible position of a target in a short period time by using the fitting of a polynomial, and the safe domain of the moving target, which is a denied area for the hunting AUVs, is built to avoid the target's escape when it detects AUVs. Secondly, the method of negotiation is adopted to allocate appropriate desired hunting points for each AUV. Finally, the AUVs arrive at desired hunting points rapidly through deep reinforcement learning (DRL) algorithm to achieve hunting the moving target. The simulations show that hunting AUVs can surround the moving target of which the trajectory is unknown rapidly and accurately by the algorithm in the 3D environment with complex obstacles and results obtained is satisfactory. INDEX TERMS Multi-AUV hunting, dynamic prediction, deep reinforcement learning, desired hunting point XIANG CAO was born in Sichuan, China. He received the B.Sc. degree in electronic and information engineering from Southwest University, Chongqing, China, in 2004, and the M.Sc. degree in communication and information systems from

show abstract

Intelligent controller for passivity-based biped robot using deep Q network

Cited by 12 publications

References 28 publications

A hybrid chaotic controller integrating hip stiffness modulation and reinforcement learning-based torque control to stabilize passive dynamic walking

A hybrid chaotic controller integrating hip stiffness modulation and reinforcement learning-based torque control to stabilize passive dynamic walking

A Disturbance Rejection Control Method Based on Deep Reinforcement Learning for a Biped Robot

Hunting Algorithm for Multi-AUV Based on Dynamic Prediction of Target Trajectory in 3D Underwater Environment

Contact Info

Product

Resources

About