Neuronlike adaptive elements that can solve difficult learning control problems

Barto, Andrew G.; Sutton, Richard S.; Anderson, Charles W.

doi:10.1109/tsmc.1983.6313077

Cited by 2,570 publications

(1,355 citation statements)

References 0 publications

Supporting

Mentioning

1,336

Contrasting

Unclassified

Order By: Relevance

“…This reduces the number of paths needed to form a navigational map to around 10 to 30 in rough agreement with experimental results (Morris et al, 1982;. Such an approach of strategy improvement during the exploration phase is similar to the well studied models of reinforcement learning (Barto et al, 1983;Dayan, 1992) and policy iteration in dynamic programming (Dayan, 1996). These approaches develop paths of minimal length by optimization whereas the case studied here involves only random paths and does not necessarily lead to maps that provide the most efficient path to the target.…”

Section: Discussionsupporting

confidence: 80%

Untitled

Gerstner

Abbott

1997

Journal of Computational Neuroscience

View full text Add to dashboard Cite

Abstract. We analyze a model of navigational map formation based on correlation-based, temporally asymmetric potentiation and depression of synapses between hippocampal place cells. We show that synaptic modification during random exploration of an environment shifts the location encoded by place cell activity in such a way that it indicates the direction from any location to a fixed target avoiding walls and other obstacles. Multiple maps to different targets can be simultaneously stored if we introduce target-dependent modulation of place cell activity. Once maps to a number of target locations in a given environment have been stored, novel maps to previously unknown target locations are automatically constructed by interpolation between existing maps.

show abstract

Section: Discussionsupporting

confidence: 80%

Untitled

Gerstner

Abbott

1997

Journal of Computational Neuroscience

View full text Add to dashboard Cite

show abstract

“…Many variants of traditional RL exist (e.g., Barto et al, 1983;Watkins, 1989;Watkins and Dayan, 1992;Moore and Atkeson, 1993;Schwartz, 1993;Rummery and Niranjan, 1994;Singh, 1994;Baird, 1995;Kaelbling et al, 1995;Peng and Williams, 1996;Mahadevan, 1996;Tsitsiklis and van Roy, 1996;Bradtke et al, 1996;Santamaría et al, 1997;Prokhorov and Wunsch, 1997;Sutton and Barto, 1998;Wiering and Schmidhuber, 1998b;Baird and Moore, 1999;Meuleau et al, 1999;Morimoto and Doya, 2000;Bertsekas, 2001;Brafman and Tennenholtz, 2002;Abounadi et al, 2002;Lagoudakis and Parr, 2003;Sutton et al, 2008;Maei and Sutton, 2010;van Hasselt, 2012). Most are formulated in a probabilistic framework, and evaluate pairs of input and output (action) events (instead of input events only).…”

Section: Deep Fnns For Traditional Rl and Markov Decision Processes (mentioning

confidence: 99%

Deep learning in neural networks: An overview

2015

View full text Add to dashboard Cite

In recent years, deep artificial neural networks (including recurrent ones) have won numerous contests in pattern recognition and machine learning. This historical survey compactly summarises relevant work, much of it from the previous millennium. Shallow and deep learners are distinguished by the depth of their credit assignment paths, which are chains of possibly learnable, causal links between actions and effects. I review deep supervised learning (also recapitulating the history of backpropagation), unsupervised learning, reinforcement learning & evolutionary computation, and indirect search for short programs encoding deep and large networks.LATEX source: http://www.idsia.ch/˜juergen/DeepLearning8Oct2014.tex Complete BIBTEX file (888 kB): http://www.idsia.ch/˜juergen/deep.bib Preface This is the preprint of an invited Deep Learning (DL) overview. One of its goals is to assign credit to those who contributed to the present state of the art. I acknowledge the limitations of attempting to achieve this goal. The DL research community itself may be viewed as a continually evolving, deep network of scientists who have influenced each other in complex ways. Starting from recent DL results, I tried to trace back the origins of relevant ideas through the past half century and beyond, sometimes using "local search" to follow citations of citations backwards in time. Since not all DL publications properly acknowledge earlier relevant work, additional global search strategies were employed, aided by consulting numerous neural network experts. As a result, the present preprint mostly consists of references. Nevertheless, through an expert selection bias I may have missed important work. A related bias was surely introduced by my special familiarity with the work of my own DL research group in the past quarter-century. For these reasons, this work should be viewed as merely a snapshot of an ongoing credit assignment process. To help improve it, please do not hesitate to send corrections and suggestions to juergen@idsia.ch.

show abstract

“…More specifically, changes in state values V MF (s) imply changes in future reward, and so a change in value induced by an action is a metric that can be used to reinforce behaviours. This forms the core of the actor-critic model (Barto et al, 1983;O'Doherty et al, 2004). Experimentally, it is perhaps most directly demonstrated by conditioned reinforcement experiments (Everitt and Robbins, 2005;Meyer et al, 2012), where instrumental behaviours can be reinforced by Pavlovian CSs.…”

Section: Instrumental Behaviourmentioning

confidence: 98%

The role of learning-related dopamine signals in addiction vulnerability

Huys

Tobler

Hasler

et al. 2014

Progress in Brain Research

View full text Add to dashboard Cite

Psychostimulants such as methylphenidate (MPH) and antidepressants such as fluoxetine (FLX) are widely used in the treatment of various mental disorders or as cognitive enhancers. These medications are often combined, for example, to treat comorbid disorders. There is a considerable body of evidence from animal models indicating that individually these psychotropic medications can have detrimental effects on the brain and behavior, especially when given during sensitive periods of brain development. However, almost no studies investigate possible interactions between these drugs. This is surprising given that their combined neurochemical effects (enhanced dopamine and serotonin neurotransmission) mimic some effects of illicit drugs such as cocaine and amphetamine. Here, we summarize recent studies in juvenile rats on the molecular effects in the mid-and forebrain and associated behavioral changes, after such combination treatments. Our findings indicate that these combined MPH + FLX treatments can produce similar molecular changes as seen after cocaine exposure while inducing behavioral changes indicative of dysregulated mood and motivation, effects that often endure into adulthood. Tables: 2 References: 254 ABSTRACT Dopaminergic signals play a mathematically precise role in reward-related learning, and variations in dopaminergic signalling have been implicated in vulnerability to addiction. Here, we provide a detailed overview of the relationship between theoretical, mathematical and experimental accounts of phasic dopamine signalling, with implications for the role of learning-related dopamine signalling in addiction and related disorders. We describe the theoretical and behavioural characteristics of model-free learning based on errors in the prediction of reward, including step-by-step explanations of the underlying equations. We then use recent insights from an animal model that highlights individual variation in learning during a Pavlovian conditioning paradigm to describe overlapping aspects of incentive salience attribution and model-free learning. We argue that this provides a computationally coherent account of some features of addiction. CONTENTS

show abstract

Neuronlike adaptive elements that can solve difficult learning control problems

Cited by 2,570 publications

References 0 publications

Untitled

Untitled

Deep learning in neural networks: An overview

The role of learning-related dopamine signals in addiction vulnerability

Contact Info

Product

Resources

About