A Deterministic Improved Q-Learning for Path Planning of a Mobile Robot

Abstract

Abstract-The paper provides a new deterministic Q-learning with a presumed knowledge about the distance from the current state to both the next state and the goal. This knowledge is efficiently used to update the entries in the Q-table once only by utilizing four derived properties of the Q-learning, instead of repeatedly updating them like the classical Q-learning. Naturally, the proposed algorithm has an insignificantly small timecomplexity in comparison to its classical counterpart. Further, the proposed algorithm stores the Q-value for the best possible action at a state, and thus saves significant storage. Experiments undertaken on simulated maze and real platforms confirm that the Q-table obtained by the proposed Q-learning when used for path-planning application of mobile robots outperforms both the classical and extended Q-learning with respect to three metrics: traversal time, number of states traversed, and 90 o turns required. Reduction in 90 o turnings minimizes the energy consumption, and thus has importance in robotics literature.

A hierarchical path planning approach based on A ⁎ and least-squares policy iteration for mobile robots

“…Konar et al [24] have applied Q-learning to path planning of a mobile robot. However, if the state spaces become very large or continuous, these algorithms will be computation expensive and impractical for applications.…”

“…The other contribution of this paper is the application of value function approximation with linear architecture in path planning of mobile robots. Tabularbased reinforcement learning methods, like Q-learning, have been applied in path planning with discrete state spaces [24]. However, little work has been done to use linear function approximation methods to deal with the problem of path planning in large or continuous state spaces.…”

Error Bound Analysis of $Q$ -Function for Discounted Optimal Control Problems With Policy Iteration

“…μ 0 is an admissible control policy and Q μ 0 is the Q-function of μ 0 . The approximate Q-functionQμ i satisfies the iterative error condition (38). Then, the Q-function sequence {Qμ i } approaches Q * according to the following inequalities:…”

Reinforcement Learning endowed with safe veto policies to learn the control of Linked-Multicomponent Robotic Systems

“…The success of Q-Learning has lead to many applications, such as path planning [25,31], energy management [30], routing in vehicular ad-hoc networks [42], management of water resources [27], and production planning [10].…”