Reinforcement Learning-Based Coverage Path Planning with Implicit Cellular Decomposition

Abstract

Coverage path planning in a generic known environment is shown to be NP-hard. When the environment is unknown, it becomes more challenging as the robot is required to rely on its online map information built during coverage for planning its path. A significant research effort focuses on designing heuristic or approximate algorithms that achieve reasonable performance. Such algorithms have sub-optimal performance in terms of covering the area or the cost of coverage, e.g., coverage time or energy consumption. In this paper, we provide a systematic analysis of the coverage problem and formulate it as an optimal stopping time problem, where the trade-off between coverage performance and its cost is explicitly accounted for. Next, we demonstrate that reinforcement learning (RL) techniques can be leveraged to solve the problem computationally. To this end, we provide some technical and practical considerations to facilitate the application of the RL algorithms and improve the efficiency of the solutions. Finally, through experiments in grid world environments and Gazebo simulator, we show that reinforcement learning-based algorithms efficiently cover realistic unknown indoor environments, and outperform the current state of the art.

UAV Coverage Path Planning under Varying Power Constraints using Deep Reinforcement Learning

“…In [14], the authors develop an abstraction model for CPP scenarios to find a specific coverage path solution with DRL. In [17], a CPP problem is reformulated as an optimal time-stopping problem, and the authors demonstrate a solution to this problem with a deep Q-network approach for indoor environments. Kyaw et al [15] use a classical cellular decomposition approach to repose the CPP problem as a traveling salesman problem, then solve it with the help of the REINFORCE algorithm.…”

“…This ablation study explores the impacts of different action masks on the learning performance. Multi3 agents (i.e., agents trained on three maps, see Table I) were trained using either no mask, the valid mask ( 14), the immediate mask (15), or the invariant mask (17). For the purpose of this comparison, agents that violated a constraint received a penalty of r s = 5 (r c = 0.01 and r m = 0.02), and the episode was terminated.…”

A reinforcement learning (RL)-based hybrid method for ground penetrating radar (GPR)-driven buried object detection

“…We note that, for such sensor-based path planning and/or target detection problems, machine learning-based methods have been on the agenda recently. In particular, reinforcement learning (RL) approach appears to be gaining popularity as an adaptive optimization methodology [8][9][10][11][12].…”

Advanced Power Converters and Learning in Diverse Robotic Innovation: A Review

“…In the service industry, AI-driven robots are employed in customer interactions, from chatbots resolving inquiries to robots providing room service in hotels. Household robotics, such as smart vacuum cleaners, leverage machine learning to map and clean spaces effectively [16][17][18].…”