Dual-Arm Robot Trajectory Planning Based on Deep Reinforcement Learning under Complex Environment

Abstract

In this article, the trajectory planning of the two manipulators of the dual-arm robot is studied to approach the patient in a complex environment with deep reinforcement learning algorithms. The shape of the human body and bed is complex which may lead to the collision between the human and the robot. Because the sparse reward the robot obtains from the environment may not support the robot to accomplish the task, a neural network is trained to control the manipulators of the robot to prepare to hold the patient up by using a proximal policy optimization algorithm with a continuous reward function. Firstly, considering the realistic scene, the 3D simulation environment is built to conduct the research. Secondly, inspired by the idea of the artificial potential field, a new reward and punishment function was proposed to help the robot obtain enough rewards to explore the environment. The function is consisting of four parts which include the reward guidance function, collision detection, obstacle avoidance function, and time function. Where the reward guidance function is used to guide the robot to approach the targets to hold the patient, the collision detection and obstacle avoidance function are complementary to each other and are used to avoid obstacles, and the time function is used to reduce the number of training episode. Finally, after the robot is trained to reach the targets, the training results are analyzed. Compared with the DDPG algorithm, the PPO algorithm reduces about 4 million steps for training to converge. Moreover, compared with the other reward and punishment functions, the function used in this paper will obtain many more rewards at the same training time. Apart from that, it will take much less time to converge, and the episode length will be shorter; so, the advantage of the algorithm used in this paper is verified.

Robot Trajectory Planning Based on the Energy Management Strategy

“…At present, mobile robot functions have been more perfect, and product types have become more and more abundant, such as the service mobile robot PR2, which was developed by Stanford Research Institute and designed and manufactured by Willow Garage in 2010 [15]; several mobile robots applied to service, education, industry, and other fields were independently developed by Harbin University Robotics Group [16]; the first mobile robot was developed by Boston. e first bionic quadruped mobile robot that can be applied to complex scenarios, Big Dog, was developed by Boston Dynamics in 2005 [17].…”

Motion planning framework based on dual-agent DDPG method for dual-arm robots guided by human joint angle constraints

“…In the study by Jiang et al ( 2021 ), a multiagent twin delayed deep deterministic policy gradient (MATD3) algorithm was proposed for the on-orbit acquisition mission of a space robot arm to generate a real-time inverse kinematics solution for the coordinated robot arm. In the study by Tang et al ( 2022 ), the proximal policy optimization (PPO) algorithm with continuous rewards was used for trajectory planning of the two-arm robot, and the reward and punishment function was designed based on the artificial potential field (APF) method so that the dual-arm robot could approach and support patients in a complex environment. This category method imitates human beings from the level of learning style and mimics the human trial and error reward learning mechanism.…”

A Self-Collision Detection Algorithm of a Dual-Manipulator System Based on GJK and Deep Learning

“…Robots, especially manipulators, now play a significant part in medical [ 1 , 2 ], aerospace [ 3 , 4 ], industrial production [ 5 , 6 ], and other industries as a result of the ongoing advancements in science and technology, helping people solve problems by delivering distinct advantages. In recent years, with a deepening of the application of manipulators in various fields, the complexity of tasks has gradually increased, and many tasks require the cooperative operation of dual or multiple manipulators, such as the extraction and transportation of heavy objects [ 7 ].…”