Reinforcement Learning in Robotics: A Survey

Abstract

Reinforcement learning offers to robotics a framework and set of tools for the design of sophisticated and hard-to-engineer behaviors. Conversely, the challenges of robotic problems provide both inspiration, impact, and validation for developments in reinforcement learning. The relationship between disciplines has sufficient promise to be likened to that between physics and mathematics. In this article, we attempt to strengthen the links between the two research communities by providing a survey of work in reinforcement learning for behavior generation in robots. We highlight both key challenges in robot reinforcement learning as well as notable successes. We discuss how contributions tamed the complexity of the domain and study the role of algorithms, representations, and prior knowledge in achieving these successes. As a result, a particular focus of our paper lies on the choice between modelbased and model-free as well as between value function-based and policy search methods. By analyzing a simple problem in some detail we demonstrate how reinforcement learning approaches may be profitably applied, and we note throughout open questions and the tremendous potential for future research.

Deep Active Learning for Autonomous Navigation

“…A sequence of raw frames is used as input to the network and trial and error is used to learn a policy. Trial and error methods such as reinforcement learning have been extensively used to learn policies for intelligent agents [17]. However, providing demonstrations of correct behavior can greatly expedite the learning rate.…”

An Intelligent Control System for Mobile Robot Navigation Tasks in Surveillance

“…Reinforcement learning is a promising approach to deal with control of physical robot with ever increasing complexity of hardware [22], [23] through experience and observations. Q-learning algorithm is a popular model-free reinforcement learning that have been demonstrated to give good results for some instances of robot tasks over the years.…”

“…It has been widely applied to the design of robot speed and orientation steering controller because of the following reasons: 1) Control rules are more flexible, thus it can simplify the complex system; 2) The controller can emulate the human decision making; 3) It does not need a detailed model of the plant, and it replaces the mathematical values in describing control system by using the linguistic ambiguous labels for designing robust controllers. On the other hand, reinforcement learning, in particular Q-learning, shows good learning results in designing control input for performing constrained tasks by robots without knowing the system dynamics [22], [23]. The approaches of combining type-1 fuzzy logic and Q-learning for optimization of the consequence parts of fuzzy rules are promising due to the ease of implementation on mobile robot navigation [12]- [17] in which Q value is a cost for each navigation behavior.…”

Layered direct policy search for learning hierarchical skills

“…Many HRL methods have been proposed in order to reduce the complexity of the task [1]- [4]. The HAM framework [5] can learn complex hierarchical sub-routines.…”