On diffusion processes with variable drift rates as models for decision making during learning

Eckhoff, Philip A.; Holmes, Philip; Law, Chi-Tat; Connolly, Patrick; Gold, Joshua I.

doi:10.1088/1367-2630/10/1/015006

Cited by 40 publications

(50 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…An instance of this framework is the learning of reaction times when detecting coherent motion within a random motion field [18]. An analysis of behavioral data [19] indicates that performance improvements in this task largely result from an increase in the evidence rate. So the input into the decision making circuitry becomes more reliable, as it would in our population learning.…”

Section: Discussionmentioning

confidence: 99%

Reinforcement learning in populations of spiking neurons

2009

View full text Add to dashboard Cite

Population coding is widely regarded as a key mechanism for achieving reliable behavioral responses in the face of neuronal variability. But in standard reinforcement learning a flip-side becomes apparent. Learning slows down with increasing population size since the global reinforcement becomes less and less related to the performance of any single neuron. We show that, in contrast, learning speeds up with increasing population size if feedback about the population response modulates synaptic plasticity in addition to global reinforcement. The two feedback signals (reinforcement and population-response signal) can be encoded by ambient neurotransmitter concentrations which vary slowly, yielding a fully online plasticity rule where the learning of a stimulus is interleaved with the processing of the subsequent one. The assumption of a single additional feedback mechanism therefore reconciles biological plausibility with efficient learning.

show abstract

Section: Discussionmentioning

confidence: 99%

Reinforcement learning in populations of spiking neurons

2009

View full text Add to dashboard Cite

show abstract

“…Behavioral data can often be explained reasonably well by a wide range of models (Bogacz et al 2006;Churchland et al 2008;Deneve 2012;Ditterich 2006bDitterich , 2010Drugowitsch et al 2012;Eckhoff et al 2008;Huang and Rao 2013;Palmer et al 2005;Ratcliff and Smith 2004), even when applied to the same real (Ditterich 2006a(Ditterich , 2006bDrugowitsch et al 2012;Huang andRao 2013 Mazurek et al 2003;Palmer et al 2005;Roitman and Shadlen 2002) or simulated data (Donkin et al 2011). Instead, our objective was to assess to what degree the performance of our subjects using these multicolored DC stimuli was consistent with the performance of subjects in a range of previous decision-making tasks, especially those using RDK stimuli.…”

Section: Model Simulationsmentioning

confidence: 99%

Reach target selection in humans using ambiguous decision cues containing variable amounts of conflicting sensory evidence supporting each target choice

Coallier

Kalaska

2014

Journal of Neurophysiology

View full text Add to dashboard Cite

show abstract

“…The remaining parameters were set to values used previously (m ϭ 1.25, ϭ 0.3, and r 0 ϭ 10 spikes/s; see Eckhoff et al 2008;Shadlen 2000, 2003).…”

Section: Psychometric Functionmentioning

confidence: 99%

“…Biases were assumed to correspond to offsets to the initial (or, equivalently, final) positions of the drift-diffusion process by adding an additional term to Eq. 2 (Eckhoff et al 2008)…”

Section: Psychometric Functionmentioning

confidence: 99%

The Relative Influences of Priors and Sensory Evidence on an Oculomotor Decision Variable During Perceptual Learning

Gold¹,

Law²,

Connolly³

et al. 2008

Journal of Neurophysiology

116

152

View full text Add to dashboard Cite

Gold JI, Law C-T, Connolly P, Bennur S. The relative influences of priors and sensory evidence on an oculomotor decision variable during perceptual learning. J Neurophysiol 100: 2653-2668. First published August 27, 2008 doi:10.1152/jn.90629.2008. Choice behavior on simple sensory-motor tasks can exhibit trial-to-trial dependencies. For perceptual tasks, these dependencies reflect the influence of prior trials on choices that are also guided by sensory evidence, which is often independent across trials. Here we show that the relative influences of prior trials and sensory evidence on choice behavior can be shaped by training, such that prior influences are strongest when perceptual sensitivity to the relevant sensory evidence is weakest and then decline steadily as sensitivity improves. We trained monkeys to decide the direction of random-dot motion and indicate their decision with an eye movement. We characterized sequential dependencies by relating current choices to weighted averages of prior choices. We then modeled behavior as a drift-diffusion process, in which the weighted average of prior choices provided an additive offset to a decision variable that integrated incoming motion evidence to govern choice. The average magnitude of offset within individual training sessions declined steadily as the quality of the integrated motion evidence increased over many months of training. The trial-by-trial magnitude of offset was correlated with signals related to developing commands that generate the oculomotor response but not with neural activity in either the middle temporal area, which represents information about the motion stimulus, or the lateral intraparietal area, which represents the sensory-motor conversion. The results suggest that training can shape the relative contributions of expectations based on prior trends and incoming sensory evidence to select and prepare visually guided actions. I N T R O D U C T I O NPerformance on trial-based sensory-motor tasks can exhibit numerous forms of sequential dependence. For example, both response times and choice are sensitive to effects of the previous trial, including repeated actions, task switching, and errors (Botvinick et al. 2001;Cho et al. 2002;Laming 1979Laming 1968Luce 1986). For tasks in which reward probability depends on sequential patterns of choices, these dependencies can reflect rational strategies for choosing future outcomes based on prior history Behrens et al. 2007;Corrado et al. 2005;Davidson and McCarthy 1988;Herrnstein 1961;Kennerley et al. 2006;Lau and Glimcher 2005;Sugrue et al. 2004;Williams 1988). In contrast, for perceptual tasks in which the present stimulus is the exclusive factor determining the rewarded choice, these dependencies can only hinder performance. The goal of this study was to better understand how these dependencies evolve as perceptual sensitivity to sensory cues that instruct the rewarded choice improves with training.We trained monkeys on a one-interval, two-alternative direction-discrimination task in which they d...

show abstract

On diffusion processes with variable drift rates as models for decision making during learning

Cited by 40 publications

References 32 publications

Reinforcement learning in populations of spiking neurons

Reinforcement learning in populations of spiking neurons

Reach target selection in humans using ambiguous decision cues containing variable amounts of conflicting sensory evidence supporting each target choice

The Relative Influences of Priors and Sensory Evidence on an Oculomotor Decision Variable During Perceptual Learning

Contact Info

Product

Resources

About