A Computational Framework for Automatic Online Path Generation of Robotic Inspection Tasks via Coverage Planning and Reinforcement Learning

In the research of multi-robot systems, multi-AUV (multiple autonomous underwater vehicles) cooperative target hunting is a hot issue. In order to improve the target hunting efficiency of multi-AUV, a multi-AUV hunting algorithm based on dynamic prediction for the trajectory of the moving target is proposed in this paper. Firstly, with moving of the target, sample points are updated dynamically to predict the possible position of a target in a short period time by using the fitting of a polynomial, and the safe domain of the moving target, which is a denied area for the hunting AUVs, is built to avoid the target's escape when it detects AUVs. Secondly, the method of negotiation is adopted to allocate appropriate desired hunting points for each AUV. Finally, the AUVs arrive at desired hunting points rapidly through deep reinforcement learning (DRL) algorithm to achieve hunting the moving target. The simulations show that hunting AUVs can surround the moving target of which the trajectory is unknown rapidly and accurately by the algorithm in the 3D environment with complex obstacles and results obtained is satisfactory. INDEX TERMS Multi-AUV hunting, dynamic prediction, deep reinforcement learning, desired hunting point XIANG CAO was born in Sichuan, China. He received the B.Sc. degree in electronic and information engineering from Southwest University, Chongqing, China, in 2004, and the M.Sc. degree in communication and information systems from

Section: A Target Hunting When Auv Is the Same Velocity As The Targetmentioning

confidence: 99%

Section: B Target Hunting When the Velocity Of Auv Is Slower Than Thmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Hunting Algorithm for Multi-AUV Based on Dynamic Prediction of Target Trajectory in 3D Underwater Environment

Cao

2020

“…Using deep RL, they replaced most steps of the previous pre-planning smoothing method with a neural-network based method and achieved the development of an algorithm involving fewer calculations than the usual solution. Jing et al [25] proposed a computational framework for robot path generation for surface/shape inspection application. They used an RL-based tree search algorithm to efficiently generate online paths based on the proposed MDP formulation to solve the coverage planning problem [11].…”

Section: B Related Workmentioning

confidence: 99%

Reinforcement Learning-Based Path Generation Using Sequential Pattern Reduction and Self-Directed Curriculum Learning

Kim

Lee

2020

Recent advancements in robots and deep learning have led to active research in human-robot interaction. However, non-physical interaction using visual devices such as laser pointers has gained less attention than physical interaction using complex robots such as humanoids. Such vision-based interaction has high potential for use in recent human-robot collaboration environments such as assembly guidance, even with a minimum amount of configuration. In this paper, we introduce a simple robotic laser pointer device that follows an arbitrary planar path and is designed to be a visual instructional aid. We also propose an image-based automatic path generation method using reinforcement learning and a sequential pattern reduction technique. However, such vision-based human-robot interaction is generally performed in a dynamic environment, and it can frequently be necessary to calibrate the devices more than once. In this paper, we avoid the need for this re-calibration process through episodic randomization learning and improved learning efficiency. In particular, contrary to previous approaches, the agent controls the curriculum difficulty in a self-directed manner to determine the optimal curriculum. To our knowledge, this is the first study of curriculum learning that incorporates an explicit learning environment control signal initiated by the agent itself. Through quantitative and qualitative analyses, we show that the proposed selfdirected curriculum learning method outperforms ordinary episodic randomization and curriculum learning. We hope that the proposed method can be extended to a general reinforcement learning framework. INDEX TERMS Path generation, robotic laser pointer, deep reinforcement learning, curriculum learning I. INTRODUCTION A. BACKGROUND VOLUME 0

“…When robot performs reinforcement learning, it has to be able to generalize to new actions to become autonomous. Most approaches concern that robots derive efficient representations from high-dimensional sensory inputs, and use these to generalize past experience to new situations [7] [8]. Generally, this is a hard problem, since the search space the robot has to explore is potentially huge [9].…”

Section: Introductionmentioning

confidence: 99%

A Reinforcement Learning-Based Framework for Robot Manipulation Skill Acquisition

Liu

Wang

et al. 2020

This paper studies robot manipulation skill acquisition based on a proposed reinforcement learning framework. Robot can learn policy autonomously by interacting with environment with a better learning efficiency. Aiming at the manipulator operation task, a reward function design method based on objects configuration matching (OCM) is proposed. It is simple and suitable for most Pick and Place skills learning. Integrating robot and object state, high-level action set and the designed reward function, the Markov model of robot manipulator is built. An improved Proximal Policy Optimize algorithm with manipulation set as the output of Actor (MAPPO) is proposed as the main structure to construct the robot reinforcement learning framework. The framework combines with the Markov model to learn and optimize the skill policy. A same simulation environment as the real robot is set up, and three robot manipulation tasks are designed to verify the effectiveness and feasibility of the reinforcement learning framework for skill acquisition.