Reinforcement learning can account for associative and perceptual learning on a visual-decision task

Law, Chi-Tat; Gold, Joshua I.

doi:10.1038/nn.2304

Cited by 257 publications

(314 citation statements)

References 48 publications

Supporting

Mentioning

281

Contrasting

Order By: Relevance

“…This is consistent with recent psychophysical studies showing that task learning followed by exposure to visual stimulus enables transfer of perceptual learning (Zhang et al, 2010). Generalization of learning of this type is difficult to explain in a strict reinforcement learning framework (Sutton and Barto, 1998;Law and Gold, 2009) or with the Hebbian rule (Hebb, 1949) because MT neurons in the untrained hemifield were not activated. Hence, studies aimed to understand the mechanism underlying generalization and additional learning in the untrained visual field merits future investigation.…”

Section: Implication For Long-term Learningsupporting

confidence: 89%

Change in Choice-Related Response Modulation in Area MT during Learning of a Depth-Discrimination Task is Consistent with Task Learning

2012

View full text Add to dashboard Cite

What are the neural mechanisms underlying improvement in perceptual performance due to learning? A recent study using motiondirection discrimination suggested that perceptual learning is due to improvements in the "readout" of sensory signals in sensory-motor cortex and not to improvements in neural sensitivity of the sensory cortex. To test the generality of this hypothesis, we examined this in a similar but different task. We recorded from isolated neurons in the middle temporal (MT) area while monkeys were trained in a depth-discrimination task. Consistent with earlier reports using direction discrimination, we found no long-term improvement in MT neuron sensitivity to depth, although monkey performance improved over months with extensive training, even when taking out the effect of behavioral biases. We further addressed improvement in the readout of sensory signals by focusing on choice-related response modulation [choice probability (CP)]. CP increased with training, suggesting an improvement in the readout of sensory signals from MT. CP, however, correlated more strongly with lapse rate than psychophysical threshold, suggesting that changes in readout may be restricted to early phases of learning. To test how behavioral learning, as well as the magnitude of CP, transferred across visual fields, we measured CP variation in one hemifield after training monkeys on the depth-discrimination task in the opposite hemifield. CP was large from the beginning of training in the untrained hemifield, even though a small but significant improvement in sensitivity was observed behaviorally. Overall, our findings are consistent with the idea that increases in CP reflect task learning.

show abstract

Section: Implication For Long-term Learningsupporting

confidence: 89%

Change in Choice-Related Response Modulation in Area MT during Learning of a Depth-Discrimination Task is Consistent with Task Learning

2012

View full text Add to dashboard Cite

show abstract

“…5 and 22)-our model included in the mixture a set of features that allowed describing any rule, i.e., a spanning set of features. This features set could correspond to neural correlates of decision-making (23,24), probabilistic inference (13,25), and learning and strategy shifts (26,27) that were observed in single-unit recordings in primate and mammalian cortex. Seeking neural correlates of the model presented here would be of particular interest in light of the characterization of the role of memory systems involved in WP (28,29) and other learning and decision-making tasks (30,31), and theoretical models of incremental learning through spike timing-dependent plasticity (32,33).…”

Section: Discussionmentioning

confidence: 99%

High-order feature-based mixture models of classification learning predict individual learning curves and enable personalized teaching

Cohen

Schneidman

2012

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

Pattern classification learning tasks are commonly used to explore learning strategies in human subjects. The universal and individual traits of learning such tasks reflect our cognitive abilities and have been of interest both psychophysically and clinically. From a computational perspective, these tasks are hard, because the number of patterns and rules one could consider even in simple cases is exponentially large. Thus, when we learn to classify we must use simplifying assumptions and generalize. Studies of human behavior in probabilistic learning tasks have focused on rules in which pattern cues are independent, and also described individual behavior in terms of simple, single-cue, feature-based models. Here, we conducted psychophysical experiments in which people learned to classify binary sequences according to deterministic rules of different complexity, including high-order, multicue-dependent rules. We show that human performance on such tasks is very diverse, but that a class of reinforcement learning-like models that use a mixture of features captures individual learning behavior surprisingly well. These models reflect the important role of subjects' priors, and their reliance on high-order features even when learning a low-order rule. Further, we show that these models predict future individual answers to a high degree of accuracy. We then use these models to build personally optimized teaching sessions and boost learning.inference | information | maximum entropy W e regularly learn to classify sensory stimuli into novel categories, often using an impoverished sampling of the stimulus space and the underlying rule: even a "simple" task of classifying patterns of n bits into two categories, requires an implicit mapping of the 2 n possible patterns, which means there are 2 2 n potential deterministic classification rules. It is clear then that when we learn to classify, we cannot simply explore the space of rules and patterns, but instead must rely on simplifying assumptions.Analysis of human learning of deterministic classification rules has focused on modeling of the average behavior of subjects (1-3), and explored the effect of rule complexity on the average level of success (4). Learning to classify according to a probabilistic rule is inherently ambiguous, and so studies of such tasks have focused on simpler rules than those used in deterministic classification. For example, the weather prediction (WP) task (5) requires learning probabilistic associations between multiple cues and a label, where each cue carries independent information about the correct label. Analysis of the learning strategy of individual subjects in this task has compared single-cue or single feature-based strategies, and the possibility of switching between such strategies (6, 7). Associative or Bayesian learning models that rely on simple stimulus features were used to describe the diversity of individual learning dynamics that subjects exhibited and compare between subjects (8), and reflected differences between healthy sub...

show abstract

“…4 A, see Materials and Methods). The structure of each detector channel was similar to that of past models that simulated a single pool of MT neurons in a motion discrimination task Law and Gold, 2009). The summated output of the pool of model neurons of each detector channel was integrated in time to detect a 50 ms motion signal that could occur in either both pools simultaneously or just one pool.…”

Section: A Feedforward Model Reproduces Neural-behavioral Correlationsmentioning

confidence: 96%

“…The sensitivity of a visual neuron in the dorsal stream has been shown to be proportional to its neural-behavioral correlations (Celebrini and Newsome, 1994;Britten et al, 1996;Shadlen et al, 1996;Cook and Maunsell, 2002;Parker et al, 2002;Uka and DeAngelis, 2004;Purushothaman and Bradley, 2005;Gu et al, 2008;Law and Gold, 2008;Ghose and Harrison, 2009;Law and Gold, 2009;Price and Born, 2010). In other words, the most reliable neurons at signaling the behaviorally relevant stimulus also tend to be the neurons that best predict behavioral performance.…”

Section: Neural Sensitivitymentioning

confidence: 99%

“…A complicating factor in this causal interpretation is that fluctuations in neural activity may arise from different sources and at different times relative to the perceptual decision Uka and DeAngelis, 2004;Nienborg and Cumming, 2007;Gu et al, 2008;Law and Gold, 2008;Sasaki and Uka, 2009). For example, neural-behavioral covariations may depend on the strength of interneuron correlations Cohen and Newsome, 2009;Law and Gold, 2009;Nienborg and Cumming, 2010), which can change with task or attentional demands (Cohen and Newsome, 2008;Cohen and Maunsell, 2009;Law and Gold, 2009;Mitchell et al, 2009;Churchland et al, 2010). Nienborg and Cumming (2009) provided a particularly clear example of the potential non-causal component of neural-behavioral correlations by showing that they increase beyond the likely time a perceptual decision was made.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The Functional Link between Area MT Neural Fluctuations and Detection of a Brief Motion Stimulus

Smith

Zhan²,

Cook³

2011

J. Neurosci.

View full text Add to dashboard Cite

Fluctuations of neural firing rates in visual cortex are known to be correlated with variations in perceptual performance. It is important to know whether these fluctuations are functionally linked to perception in a causal manner or instead reflect non-causal processes that arise after the perceptual decision is made. We recorded from middle temporal (MT) neurons from monkey subjects while they detected the random occurrence of a brief 50 ms motion pulse that occurred in either of two (or simultaneously in both) random dot patches located in the same hemisphere. The receptive field parameters of the motion pulse were matched to that preferred by each MT neuron under study. This task contained uncertainty in both space and time because, on any given trial, the subjects did not know which patch would contain the motion pulse or when the motion pulse would occur. Covariations between MT activity and behavior began just before the motion pulse onset and peaked at the maximum neural response. These neural-behavioral covariations were strongest when only one patch contained the motion pulse and were still weakly present when a patch did not contain a motion pulse. A feedforward temporal integration model with two independent detector channels captured both the detection performance and evolution of the neuralbehavior covariations over time and stimulus condition. The results suggest that, when detecting a brief visual stimulus, there is a causal relationship between fluctuations in neural activity and variations in behavior across trials.

show abstract

Reinforcement learning can account for associative and perceptual learning on a visual-decision task

Cited by 257 publications

References 48 publications

Change in Choice-Related Response Modulation in Area MT during Learning of a Depth-Discrimination Task is Consistent with Task Learning

Change in Choice-Related Response Modulation in Area MT during Learning of a Depth-Discrimination Task is Consistent with Task Learning

High-order feature-based mixture models of classification learning predict individual learning curves and enable personalized teaching

The Functional Link between Area MT Neural Fluctuations and Detection of a Brief Motion Stimulus

Contact Info

Product

Resources

About