Reinforcement Learning is Direct Adaptive Optimal Control

Sutton, Richard S.; Barto, Andrew G.; Williams, Ronald J.

doi:10.23919/acc.1991.4791776

Cited by 267 publications

(356 citation statements)

References 17 publications

Supporting

Mentioning

341

Contrasting

Unclassified

Order By: Relevance

“…More details can be found in [21]. A derivative of supervised learning is so-called reinforcement learning, which is based trail and error (and reward) [68] and has backings from psychology.…”

Section: Lateral Inhibition and Hebbian Learningmentioning

confidence: 99%

Learning Nonlinear Principal Manifolds by Self-Organising Maps

Yin

2008

Lecture Notes in Computational Science and Enginee

View full text Add to dashboard Cite

Summary. This chapter provides an overview on the self-organised map (SOM) in the context of manifold mapping. It first reviews the background of the SOM and issues on its cost function and topology measures. Then its variant, the visualisation induced SOM (ViSOM) proposed for preserving local metric on the map, is introduced and reviewed for data visualisation. The relationships among the SOM, ViSOM, multidimensional scaling, and principal curves are analysed and discussed. Both the SOM and ViSOM produce a scaling and dimension-reduction mapping or manifold of the input space. The SOM is shown to be a qualitative scaling method, while the ViSOM is a metric scaling and approximates a discrete principal curve/surface. Examples and applications of extracting data manifolds using SOM-based techniques are presented.Key words: Self-organising maps, principal curve and surface, data visualisation, topographic mapping IntroductionFor many years, artificial neural networks have been studied and used to construct information processing systems based on or inspired by natural biological neural structures. They not only provide solutions with improved performance when compared with traditional problem-solving methods, but also give a deeper understanding of human cognitive abilities. Among the various existing neural network architectures and learning algorithms, Kohonen's self-organising map (SOM) [35] is one of most popular neural network models. Developed for an associative memory model, it is an unsupervised learning algorithm with simple structures and computational forms, and is motivated by the retina-cortex mapping. Self-organisation in general is a fundamental pattern recognition process, in which intrinsic inter-and intra-pattern relationships within the data set are learnt without the presence of a potentially biased or subjective external influence. The SOM can provide topologically

show abstract

Section: Lateral Inhibition and Hebbian Learningmentioning

confidence: 99%

Learning Nonlinear Principal Manifolds by Self-Organising Maps

Yin

2008

Lecture Notes in Computational Science and Enginee

View full text Add to dashboard Cite

show abstract

“…In contrast, reinforcement learning operates directly on measured data and rewards from interaction, and can also address cases which are analytically intractable using approximations and data-driven techniques. A concise treatment of reinforcement learning as "adaptive optimal control" is presented in [Sutton et al, 1991].…”

Section: Introductionmentioning

confidence: 99%

Learning motor skills: from algorithms to robot experiments

Kober

Peters

2014

It - Information Technology

View full text Add to dashboard Cite

Die Veröffentlichung steht unter folgender Creative Commons Lizenz: Namensnennung -Keine kommerzielle Nutzung -Keine Bearbeitung 2.0 Deutschland http://creativecommons.org/licenses/by-nc-nd/2.0/de/ Abstract Ever since the word "robot" was introduced to the English language by KarelČapek's play "Rossum's Universal Robots" in 1921, robots have been expected to become part of our daily lives. In recent years, robots such as autonomous vacuum cleaners, lawn mowers, and window cleaners, as well as a huge number of toys have been made commercially available. However, a lot of additional research is required to turn robots into versatile household helpers and companions. One of the many challenges is that robots are still very specialized and cannot easily adapt to changing environments and requirements. Since the 1960s, scientists attempt to provide robots with more autonomy, adaptability, and intelligence. Research in this field is still very active but has shifted focus from reasoning based methods towards statistical machine learning. Both navigation (i.e., moving in unknown or changing environments) and motor control (i.e., coordinating movements to perform skilled actions) are important sub-tasks.In this thesis, we will discuss approaches that allow robots to learn motor skills. We mainly consider tasks that need to take into account the dynamic behavior of the robot and its environment, where a kinematic movement plan is not sufficient. The presented tasks correspond to sports and games but the presented techniques will also be applicable to more mundane household tasks. Motor skills can often be represented by motor primitives. Such motor primitives encode elemental motions which can be generalized, sequenced, and combined to achieve more complex tasks. For example, a forehand and a backhand could be seen as two different motor primitives of playing table tennis. We show how motor primitives can be employed to learn motor skills on three different levels. First, we discuss how a single motor skill, represented by a motor primitive, can be learned using reinforcement learning. Second, we show how such learned motor primitives can be generalized to new situations. Finally, we present first steps towards using motor primitives in a hierarchical setting and how several motor primitives can be combined to achieve more complex tasks.To date, there have been a number of successful applications of learning motor primitives employing imitation learning. However, many interesting motor learning problems are high-dimensional reinforcement learning problems which are often beyond the reach of current reinforcement learning methods. We review research on reinforcement learning applied to robotics and point out key challenges and important strategies to render reinforcement learning tractable. Based on these insights, we introduce novel learning approaches both for single and generalized motor skills.For learning single motor skills, we study parametrized policy search methods and introduce a framework of reward-weighted imi...

show abstract

“…This will lead to an optimization-like problem which cannot be handled by conventional optimal control, e.g., linear quadratic regulator (LQR) [5], due to uncertain and nonlinear system dynamics. In the literature, reinforcement learning, also known as adaptive dynamic programming, has been extensively studied in the control community to address this issue [6,7].…”

Section: Introductionmentioning

confidence: 99%

Reinforcement learning control for coordinated manipulation of multi-robots

Chen

Tee

et al. 2015

Neurocomputing

View full text Add to dashboard Cite

In this paper, coordination control is investigated for multi-robots to manipulate an object with a common desired trajectory. Both trajectory tracking and control input minimization are considered for each individual robot manipulator, such that possible disagreement between different manipulators can be handled. Reinforcement learning is employed to cope with the problem of unknown dynamics of both robots and the manipulated object. It is rigorously proven that the proposed method guarantees the coordination control of the multi-robots system under study. The validity of the proposed method is verified through simulation studies.

show abstract

Reinforcement Learning is Direct Adaptive Optimal Control

Cited by 267 publications

References 17 publications

Learning Nonlinear Principal Manifolds by Self-Organising Maps

Learning Nonlinear Principal Manifolds by Self-Organising Maps

Learning motor skills: from algorithms to robot experiments

Reinforcement learning control for coordinated manipulation of multi-robots

Contact Info

Product

Resources

About