Neural networks based reinforcement learning for mobile robots obstacle avoidance

Duguleană, Mihai; Mogan, Gheorghe

doi:10.1016/j.eswa.2016.06.021

Cited by 148 publications

(46 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Q-learning and Sarsa [12] can be useful for dealing with discrete state spaces. For example, Mihai Duguleana et al [21] combined Q-learning with the artificial neural network for solving the problem of autonomous movement of robots in environments that contain both static and dynamic obstacles. However, in large or continuous state spaces, the abovementioned tabular RL methods are inefficient or impractical for applications.…”

Section: Reinforcement Learningmentioning

confidence: 99%

Combining Subgoal Graphs with Reinforcement Learning to Build a Rational Pathfinder

Zeng

Qin

et al. 2019

Applied Sciences

View full text Add to dashboard Cite

In this paper, we present a hierarchical path planning framework called SG-RL (subgoal graphs-reinforcement learning), to plan rational paths for agents maneuvering in continuous and uncertain environments. By "rational", we mean (1) efficient path planning to eliminate first-move lags;(2) collision-free and smooth for agents with kinematic constraints satisfied. SG-RL works in a two-level manner. At the first level, SG-RL uses a geometric path-planning method, i.e., Simple Subgoal Graphs (SSG), to efficiently find optimal abstract paths, also called subgoal sequences. At the second level, SG-RL uses an RL method, i.e., Least-Squares Policy Iteration (LSPI), to learn near-optimal motion-planning policies which can generate kinematically feasible and collision-free trajectories between adjacent subgoals. The first advantage of the proposed method is that SSG can solve the limitations of sparse reward and local minima trap for RL agents; thus, LSPI can be used to generate paths in complex environments. The second advantage is that, when the environment changes slightly (i.e., unexpected obstacles appearing), SG-RL does not need to reconstruct subgoal graphs and replan subgoal sequences using SSG, since LSPI can deal with uncertainties by exploiting its generalization ability to handle changes in environments. Simulation experiments in representative scenarios demonstrate that, compared with existing methods, SG-RL can work well on large-scale maps with relatively low action-switching frequencies and shorter path lengths, and SG-RL can deal with small changes in environments. We further demonstrate that the design of reward functions and the types of training environments are important factors for learning feasible policies.

show abstract

Section: Reinforcement Learningmentioning

confidence: 99%

Combining Subgoal Graphs with Reinforcement Learning to Build a Rational Pathfinder

Zeng

Qin

et al. 2019

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…During the motion process, the agent learns experience constantly [12]. The learning way is expressed as follows:…”

Section: Q-learning Algorithmmentioning

confidence: 99%

A Novel Path Planning Algorithm Based on Q-learning and Adaptive Exploration Strategy

Li¹,

Li²

2019

2019 Scientific Conference on Network, Power Systems and Computing (NPSC 2019)

View full text Add to dashboard Cite

In an unknown environment, how to plan a path is a fundamental problem for agents. In this paper, we propose an improved reinforcement learning algorithm, called adaptive exploration Q-learning algorithm (AEQ), to solve path planning problem. Firstly, to ensure an agent learns autonomously in the process of trial and error when the agent knows nothing about the environment, AEQ chooses Qlearning algorithms to improve. Secondly, AEQ utilizes an adaptive exploration strategy that aims at speeding up the algorithm's convergence. The adaptive exploration strategy dynamically adjusts the exploration factor according to various situations so that the agent explores the environment sufficiently and makes full use of the environment information. The experimental results show that the agent successfully reaches the goal without collision by AEQ. Besides, compared with the classical Q-learning algorithm and SARSA algorithm, AEQ improves the convergence speed and reduces the convergence time.

show abstract

“…Recently, neural networks have been successfully applied in path planning for mobile robots [2] [3] [4] [5]. The prospect of using recurrent neural networks increased in the last few years, since they showed to obtain the highest performances in various sequence processing tasks like speech recognition and video captioning [6] [7] [8].…”

Section: Introductionmentioning

confidence: 99%

A LSTM Neural Network applied to Mobile Robots Path Planning

Fiorato

Fujimoto

Oboe

2018

2018 IEEE 16th International Conference on Industrial Informatics (INDIN)

View full text Add to dashboard Cite

Mobile robots path planning is a central problem in every situation where human intervention is not desired or not possible to accept: full automated industrial warehouses or general stocking areas and every domestic application that involves a mobile robot and special cases where environment is prohibited for human accessing like toxic wastes and bombs defusing [1]. Currently, neural networks are applied to problems related to mobile robot navigation. However, they are not as popular as in applications like image processing, speech recognition or machine translation, where they are commercially relevant. In this paper we propose a Long Short-Term Memory (LSTM) neural network as an online search agent to tackle the problem of mobile robots path planning in unknown environments, meaning that the agent relies only on local map awareness realized with a LRF sensor and relative informations between robot and goal position. Specifically, a final structure of LSTM network is analyzed and its performance is compared with the A* algorithm, a widely known method that follows the best-first search approach. Subsequently, an analysis of the method developed on a real robot is described.

show abstract

Neural networks based reinforcement learning for mobile robots obstacle avoidance

Cited by 148 publications

References 47 publications

Combining Subgoal Graphs with Reinforcement Learning to Build a Rational Pathfinder

Combining Subgoal Graphs with Reinforcement Learning to Build a Rational Pathfinder

A Novel Path Planning Algorithm Based on Q-learning and Adaptive Exploration Strategy

A LSTM Neural Network applied to Mobile Robots Path Planning

Contact Info

Product

Resources

About