A Mapless Local Path Planning Approach Using Deep Reinforcement Learning Framework

Abstract

The key module for autonomous mobile robots is path planning and obstacle avoidance. Global path planning based on known maps has been effectively achieved. Local path planning in unknown dynamic environments is still very challenging due to the lack of detailed environmental information and unpredictability. This paper proposes an end-to-end local path planner n-step dueling double DQN with reward-based ϵ-greedy (RND3QN) based on a deep reinforcement learning framework, which acquires environmental data from LiDAR as input and uses a neural network to fit Q-values to output the corresponding discrete actions. The bias is reduced using n-step bootstrapping based on deep Q-network (DQN). The ϵ-greedy exploration-exploitation strategy is improved with the reward value as a measure of exploration, and an auxiliary reward function is introduced to increase the reward distribution of the sparse reward environment. Simulation experiments are conducted on the gazebo to test the algorithm’s effectiveness. The experimental data demonstrate that the average total reward value of RND3QN is higher than that of algorithms such as dueling double DQN (D3QN), and the success rates are increased by 174%, 65%, and 61% over D3QN on three stages, respectively. We experimented on the turtlebot3 waffle pi robot, and the strategies learned from the simulation can be effectively transferred to the real robot.

Path Planning for Autonomous Mobile Robot Using Intelligent Algorithms

“…Another journal with important contributions is Sensors from MDPI. In recent years, some papers are [38], which introduces a deep Q-network for using LiDAR data to generate discrete actions; [39], which applied a method called Hindsight Experience Replay to mitigate the sparse reward problem; and [40], which presented a technique for calculating the best path in environments with multiple robots. Their main focus was path planning in wider environments, obtaining more reliable probability tables [41].…”

An algorithm to create model file for Partially Observable Markov Decision Process for mobile robot path planning

“…These solvers have the advantage of handling large POMDPs [ 18 ], but they have the drawback of being slow compared to offline solvers [7] . The Offline solvers can arrive at better policies than online solvers for the sampled belief states, but usually, they fail to scale up to large POMDP [ 19 ]. Recently there have also been successful attempts to use Deep Reinforcement Learning techniques for solving POMDP [ 19 ].…”

“…The Offline solvers can arrive at better policies than online solvers for the sampled belief states, but usually, they fail to scale up to large POMDP [ 19 ]. Recently there have also been successful attempts to use Deep Reinforcement Learning techniques for solving POMDP [ 19 ].…”

Improved Robot Path Planning Method Based on Deep Reinforcement Learning

“…There is an increasing emphasis on autonomous robot path planning as robots are used in more and increasingly important applications [ 1 ]. In general, robot path planning algorithms are divided into two categories: traditional methods and machine learning methods [ 2 ]. Traditional methods may require extensive calculations that are difficult to meet real-time requirements or result in locally optimal solutions that fail to produce accurate paths [ 3 ].…”