Mobile Robot Path Planning in Dynamic Environments Through Globally Guided Reinforcement Learning

Wang, Binyu; Liu, Zhe; Li, Qingbiao; Prorok, Amanda

doi:10.1109/lra.2020.3026638

Cited by 165 publications

(90 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The hierarchical reinforcement learning technology is utilized to achieve the mapping from state to action and meet the mobile needs of mobile robots. The data have also proven that the robot path planning method based on deep reinforcement learning is an effective end-to-end mobile robot path planning method, which has also been confirmed in a study by Wang B. et al (2020). The above results illustrate the feasibility of the proposed method in the path planning of mobile robots.…”

Section: Discussionsupporting

confidence: 66%

The Path Planning of Mobile Robot by Neural Networks and Hierarchical Reinforcement Learning

Liao

2020

Front. Neurorobot.

View full text Add to dashboard Cite

Existing mobile robots cannot complete some functions. To solve these problems, which include autonomous learning in path planning, the slow convergence of path planning, and planned paths that are not smooth, it is possible to utilize neural networks to enable to the robot to perceive the environment and perform feature extraction, which enables them to have a fitness of environment to state action function. By mapping the current state of these actions through Hierarchical Reinforcement Learning (HRL), the needs of mobile robots are met. It is possible to construct a path planning model for mobile robots based on neural networks and HRL. In this article, the proposed algorithm is compared with different algorithms in path planning. It underwent a performance evaluation to obtain an optimal learning algorithm system. The optimal algorithm system was tested in different environments and scenarios to obtain optimal learning conditions, thereby verifying the effectiveness of the proposed algorithm. Deep Deterministic Policy Gradient (DDPG), a path planning algorithm for mobile robots based on neural networks and hierarchical reinforcement learning, performed better in all aspects than other algorithms. Specifically, when compared with Double Deep Q-Learning (DDQN), DDPG has a shorter path planning time and a reduced number of path steps. When introducing an influence value, this algorithm shortens the convergence time by 91% compared with the Q-learning algorithm and improves the smoothness of the planned path by 79%. The algorithm has a good generalization effect in different scenarios. These results have significance for research on guiding, the precise positioning, and path planning of mobile robots.

show abstract

Section: Discussionsupporting

confidence: 66%

The Path Planning of Mobile Robot by Neural Networks and Hierarchical Reinforcement Learning

Liao

2020

Front. Neurorobot.

View full text Add to dashboard Cite

show abstract

“…Wu et al successfully proposed a policy for online trajectory planning for free-floating space robot without dynamic and kinematic models. 166 Recent practical experiments 150,153,167,168 make a series of huge breakthroughs for mobile robot, sake-like robot, cleaning, and maintenance robot. These great achievements of realworld greatly promote the development of robotic applications in the future.…”

Section: Path Planningmentioning

confidence: 99%

Reinforcement learning for robot research: A comprehensive review and open issues

Zhang

2021

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

Applying the learning mechanism of natural living beings to endow intelligent robots with humanoid perception and decision-making wisdom becomes an important force to promote the revolution of science and technology in robot domains. Advances in reinforcement learning (RL) over the past decades have led robotics to be highly automated and intelligent, which ensures safety operation instead of manual work and implementation of more intelligence for many challenging tasks. As an important branch of machine learning, RL can realize sequential decision-making under uncertainties through end-to-end learning and has made a series of significant breakthroughs in robot applications. In this review article, we cover RL algorithms from theoretical background to advanced learning policies in different domains, which accelerate to solving practical problems in robotics. The challenges, open issues, and our thoughts on future research directions of RL are also presented to discover new research areas with the objective to motivate new interest.

show abstract

“…In Reference [ 15 ], well-established route planning techniques are correlated with novel neural learning approaches to find the best path to a target location within a square grid. A combination of global guidance and a local RL-based planner is presented in Reference [ 16 ]. A major downside of DRL is that such an architecture is difficult to train in real-life scenarios due to the interaction constraint and tends to generalize on specific driving scenarios (e.g., highway driving).…”

Section: Related Workmentioning

confidence: 99%

OctoPath: An OcTree-Based Self-Supervised Learning Approach to Local Trajectory Planning for Mobile Robots

Trasnea

Ginerica

Zaha

et al. 2021

Sensors

View full text Add to dashboard Cite

Autonomous mobile robots are usually faced with challenging situations when driving in complex environments. Namely, they have to recognize the static and dynamic obstacles, plan the driving path and execute their motion. For addressing the issue of perception and path planning, in this paper, we introduce OctoPath, which is an encoder-decoder deep neural network, trained in a self-supervised manner to predict the local optimal trajectory for the ego-vehicle. Using the discretization provided by a 3D octree environment model, our approach reformulates trajectory prediction as a classification problem with a configurable resolution. During training, OctoPath minimizes the error between the predicted and the manually driven trajectories in a given training dataset. This allows us to avoid the pitfall of regression-based trajectory estimation, in which there is an infinite state space for the output trajectory points. Environment sensing is performed using a 40-channel mechanical LiDAR sensor, fused with an inertial measurement unit and wheels odometry for state estimation. The experiments are performed both in simulation and real-life, using our own developed GridSim simulator and RovisLab’s Autonomous Mobile Test Unit platform. We evaluate the predictions of OctoPath in different driving scenarios, both indoor and outdoor, while benchmarking our system against a baseline hybrid A-Star algorithm and a regression-based supervised learning method, as well as against a CNN learning-based optimal path planning method.

show abstract

Mobile Robot Path Planning in Dynamic Environments Through Globally Guided Reinforcement Learning

Cited by 165 publications

References 22 publications

The Path Planning of Mobile Robot by Neural Networks and Hierarchical Reinforcement Learning

The Path Planning of Mobile Robot by Neural Networks and Hierarchical Reinforcement Learning

Reinforcement learning for robot research: A comprehensive review and open issues

OctoPath: An OcTree-Based Self-Supervised Learning Approach to Local Trajectory Planning for Mobile Robots

Contact Info

Product

Resources

About