Control and Simulation of a 6-DOF Biped Robot based on Twin Delayed Deep Deterministic Policy Gradient Algorithm

Khôi, Phan Bùi; Truong, Giang Nguyen; Tan, Hoang

doi:10.17485/ijst/v14i30.1030

Cited by 4 publications

(3 citation statements)

References 24 publications

(31 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this research, an extension of the TD3 [20] algorithm was proposed to include more information about the connection between the joints of the robot in the training process. In fact, there are many articles [20][21][22][23][24][25] using reinforcement learning algorithms such as TD3, DDPG and SAC to find the desired angle values of the joints of the robot. However, their algorithms only used the information about the velocity and angular value of the joints for training, they did not take advantage of the graph topology and the binding relationship of the humanoid robot, as in our method.…”

Section: Introductionmentioning

confidence: 99%

“…In each state, the height of the body robot has different values. In the paper [25], only an average value of the body height during motion is used as a basis height for the robot to learn, two average values of the body height corresponding to two grounding states of the legs in motion are used in this paper. At the single phase of walking (Figure 7a,b), the average height of the robot's body reaches a higher value than that at the double phase of walking (Figure 7c,d).…”

mentioning

confidence: 99%

See 1 more Smart Citation

GCTD3: Modeling of Bipedal Locomotion by Combination of TD3 Algorithms and Graph Convolutional Network

2022

View full text Add to dashboard Cite

In recent years, there has been a lot of research using reinforcement learning algorithms to train 2-legged robots to move, but there are still many challenges. The authors propose the GCTD3 method, which takes the idea of using Graph Convolutional Networks to represent the kinematic link features of the robot, and combines this with the Twin-Delayed Deep Deterministic Policy Gradient algorithm to train the robot to move. Graph Convolutional Networks are very effective in graph-structured problems such as the connection of the joints of the human-like robots. The GCTD3 method shows better results on the motion trajectories of the bipedal robot joints compared with other reinforcement learning algorithms such as Twin-Delayed Deep Deterministic Policy Gradient, Deep Deterministic Policy Gradient and Soft Actor Critic. This research is implemented on a 2-legged robot model with six independent joint coordinates through the Robot Operating System and Gazebo simulator.

show abstract

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

GCTD3: Modeling of Bipedal Locomotion by Combination of TD3 Algorithms and Graph Convolutional Network

2022

View full text Add to dashboard Cite

show abstract

“…Meanwhile, controllers based on fuzzy logic imitating human natural reasoning have been studied and applied in many control problems in general and in the control of bipedal robots-for example, a fuzzy logic-based controller for robot in mechanical processing [34,35], a fuzzy-based-admittance controller for safe natural human-robot interaction [36], and a fuzzy logic-based bipedal robot control [37][38][39]. Moreover, the integration of fuzzy logic with intelligent algorithms is also a research direction that is being applied to control bipedal robots [40][41][42].…”

Section: Introductionmentioning

confidence: 99%

Fuzzy Logic-Based Controller for Bipedal Robot

Khôi

Xuan

2021

Applied Sciences

Self Cite

View full text Add to dashboard Cite

In this paper, the problem of controlling a human-like bipedal robot while walking is studied. The control method commonly applied when controlling robots in general and bipedal robots in particular, was based on a dynamical model. This led to the need to accurately define the dynamical model of the robot. The activities of bipedal robots to replace humans, serve humans, or interact with humans are diverse and ever-changing. Accurate determination of the dynamical model of the robot is difficult because it is difficult to fully and accurately determine the dynamical quantities in the differential equations of motion of the robot. Additionally, another difficulty is that because the robot’s operation is always changing, the dynamical quantities also change. There have been a number of works applying fuzzy logic-based controllers and neural networks to control bipedal robots. These methods can overcome to some extent the uncertainties mentioned above. However, it is a challenge to build appropriate rule systems that ensure the control quality as well as the controller’s ability to perform easily and flexibly. In this paper, a method for building a fuzzy rule system suitable for bipedal robot control is proposed. The design of the motion trajectory for the robot according to the human gait and the analysis of dynamical factors affecting the equilibrium condition and the tracking trajectory were performed to provide informational data as well as parameters. Based on that, a fuzzy rule system and fuzzy controller was proposed and built, allowing a determination of the control force/moment without relying on the dynamical model of the robot. For evaluation, an exact controller based on the assumption of an accurate dynamical model, which was a two-feedback loop controller based on integrated inverse dynamics with proportional integral derivative, is also proposed. To confirm the validity of the proposed fuzzy rule system and fuzzy controller, computation and numerical simulation were performed for both types of controllers. Comparison of numerical simulation results showed that the fuzzy rule system and the fuzzy controller worked well. The proposed fuzzy rule system is simple and easy to apply.

show abstract

Learning positioning policies for mobile manipulation operations with deep reinforcement learning

Iriondo

Lazkano

Ansuategi

et al. 2023

Int. J. Mach. Learn. & Cyber.

View full text Add to dashboard Cite

This work focuses on the operation of picking an object on a table with a mobile manipulator. We use deep reinforcement learning (DRL) to learn a positioning policy for the robot’s base by considering the reachability constraints of the arm. This work extends our first proof-of-concept with the ultimate goal of validating the method on a real robot. Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm is used to model the base controller, and is optimised using the feedback from the MoveIt! based arm planner. The idea is to encourage the base controller to position itself in areas where the arm reaches the object. Following a simulation-to-reality approach, first we create a realistic simulation of the robotic environment in Unity, and integrate it in Robot Operating System (ROS). The drivers for both the base and the arm are also implemented. The DRL-based agent is trained in simulation and, both the robot and target poses are randomised to make the learnt base controller robust to uncertainties. We propose a task-specific setup for TD3, which includes state/action spaces, reward function and neural architectures. We compare the proposed method with the baseline work and show that the combination of TD3 and the proposed setup leads to a $$11\%$$ 11 % higher success rate than with the baseline, with an overall success rate of $$97\%$$ 97 % . Finally, the learnt agent is deployed and validated in the real robotic system where we obtain a promising success rate of $$75\%$$ 75 % .

show abstract

Control and Simulation of a 6-DOF Biped Robot based on Twin Delayed Deep Deterministic Policy Gradient Algorithm

Cited by 4 publications

References 24 publications

GCTD3: Modeling of Bipedal Locomotion by Combination of TD3 Algorithms and Graph Convolutional Network

GCTD3: Modeling of Bipedal Locomotion by Combination of TD3 Algorithms and Graph Convolutional Network

Fuzzy Logic-Based Controller for Bipedal Robot

Learning positioning policies for mobile manipulation operations with deep reinforcement learning

Contact Info

Product

Resources

About