Deep Reinforcement Learning-Based Path Planning for Multi-Arm Manipulators with Periodically Moving Obstacles

Abstract

In the workspace of robot manipulators in practice, it is common that there are both static and periodic moving obstacles. Existing results in the literature have been focusing mainly on the static obstacles. This paper is concerned with multi-arm manipulators with periodically moving obstacles. Due to the high-dimensional property and the moving obstacles, existing results suffer from finding the optimal path for given arbitrary starting and goal points. To solve the path planning problem, this paper presents a SAC-based (Soft actor–critic) path planning algorithm for multi-arm manipulators with periodically moving obstacles. In particular, the deep neural networks in the SAC are designed such that they utilize the position information of the moving obstacles over the past finite time horizon. In addition, the hindsight experience replay (HER) technique is employed to use the training data efficiently. In order to show the performance of the proposed SAC-based path planning, both simulation and experiment results using open manipulators are given.

Multiweight Cross-Multimedia Logistics Optimal Path Exploration by Integrating High-Dimensional Deep Learning

“…e Bohai Ring Region experienced the fastest growth in terms of infrastructure, with Tangshan port being the first with an annual growth rate of 22%, almost double that of the second largest port, Rizhao port. e rapid growth in the Bohai Rim Region is reinforcing labor competition in the region, and the negative effects of competition are evident in the area around the Yangtze River Delta, which led to a decline in overall logistics interest in China [5,6].…”

Goal-Conditioned Reinforcement Learning within a Human-Robot Disassembly Environment

“…However, the generated human motion in the collision avoidance simulation was randomized without considering possible hazardous scenarios. The same limitation can be noticed in the work of Prianto et al [39] and Sangiovanni et al [40,41]. In the former, the obstacles moved in limited trajectories periodically, while in the latter a real-time model-free collision avoidance approach was introduced and applied to robotic tasks where an unpredictable obstacle interfered with the robot's workspace.…”

“…A natural concern of the research is related to the complexity of reward shaping. While many authors provide only sparse rewards during learning [35,39], these are not usually successful when learning involves multiple goals. However, this does not necessarily imply the definition of multiple reward sources, each associated with a single goal.…”

A Deep Reinforcement Learning Strategy Combining Expert Experience Guidance for a Fruit-Picking Manipulator

“…In recent years, it has become a new solution for the problem of path planning of multi-DOF manipulators in unstructured environments, DeepMind, UC Berkeley, and many others have applied DRL to the trajectory planning problem of robotic arms [13][14][15]. Evan proposed a path-planning method for a multiarm manipulator based on the SAC (soft actor-critic) algorithm with hindsight replay (HER), which is suitable for multiarm manipulators with static and periodically mobile obstacles [16]. Chun proposed a deep reinforcement learning algorithm framework that combined the advantages of convolutional neural network (CNN) and deep deterministic policy gradient (DDPG) algorithms to solve how to use delivery task information and automated guided vehicles (AGVs) travel time in the problem of dynamic scheduling of AGV [17].…”