The mid-lateral cerebellum is necessary for reinforcement learning

Sendhilnathan, Naveen; Goldberg, Michael E.

doi:10.1101/2020.03.20.000190

Cited by 5 publications

(10 citation statements)

References 31 publications

(30 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These results show that although the mid-lateral cerebellum contributes to reinforcement learning 18 , the mechanism by which this learning occurs does not require CS-induced changes at the parallel fiber-P-cell synapse through an error-based mechanism. Rather, CS and SS form two independent channels of information, both encoding different aspects of reward-based learning depending on the context.…”

mentioning

confidence: 77%

“…When the non-human primates learn to associate arbitrary visual symbols with hand movement choices, the SS encode a reinforcement error signal during learning, which gradually diminishes through learning, and disappears once the learning is completed 17 . This error signal, which could contribute significantly to reinforcement learning 18 , is encoded as the difference in SS activity between recent correct and wrong outcomes of P-cells 17 . However, (a) the role of concurrent CS activity, (b) the interaction between SS and CS, and (c) their relationship to complex reinforcement learning and decision making are all still unknown.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Mid-lateral cerebellar complex spikes encode multiple independent reward-related signals during reinforcement learning

2021

Self Cite

View full text Add to dashboard Cite

Although the cerebellum has been implicated in simple reward-based learning recently, the role of complex spikes (CS) and simple spikes (SS), their interaction and their relationship to complex reinforcement learning and decision making is still unclear. Here we show that in a context where a non-human primate learned to make novel visuomotor associations, classifying CS responses based on their SS properties revealed distinct cell-type specific encoding of the probability of failure after the stimulus onset and the non-human primate’s decision. In a different context, CS from the same cerebellar area also responded in a cell-type and learning independent manner to the stimulus that signaled the beginning of the trial. Both types of CS signals were independent of changes in any motor kinematics and were unlikely to instruct the concurrent SS activity through an error based mechanism, suggesting the presence of context dependent, flexible, multiple independent channels of neural encoding by CS and SS. This diversity in neural information encoding in the mid-lateral cerebellum, depending on the context and learning state, is well suited to promote exploration and acquisition of wide range of cognitive behaviors that entail flexible stimulus-action-reward relationships but not necessarily motor learning.

show abstract

mentioning

confidence: 77%

mentioning

confidence: 99%

Mid-lateral cerebellar complex spikes encode multiple independent reward-related signals during reinforcement learning

2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…Both areas show signals with mixed selectivity, the cortex multiplexing selectivity for the symbol and the movement (Rigotti et al, 2013), the cerebellum multiplexing signals for learning error and state. Nonetheless reinforcement learning is impaired by inactivation of prefrontal cortex (Murray et al, 2000) as well as the mid-lateral cerebellum (Sendhilnathan and Goldberg, 2020). How these different aspects of mixed selectivity enable reinforcement learning is a puzzle that remains to be solved.…”

Section: Discussionmentioning

confidence: 99%

“…When monkeys learn to associate an arbitrary symbol with a well-learned movement in a reinforcement learning paradigm, Purkinje cell (P-cell) simple spikes produce an error signal, which decreases as the monkeys learn the task (Sendhilnathan et al, 2020). Transient inactivation of this area impaired the monkeys’ ability to learn a new set of associations (Sendhilnathan and Goldberg, 2020).…”

Section: Introductionmentioning

confidence: 99%

Mixed selectivity in the cerebellar Purkinje-cell response during visuomotor association learning

Sendhilnathan

Ipata

Goldberg

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Although the cerebellum has been traditionally considered to be exclusively involved in motor control, recent anatomical and clinical studies show that it also has a role in reward processing. However, the way in which the movement related and the reward related neural activity interact at the level of the cerebellar cortex and contribute towards learning is still unclear. Here, we studied the simple spike activity of Purkinje cells in the mid-lateral cerebellum when monkeys learned to associate a right or left hand movement with one of two visual symbolic cues. These cells had distinctly different discharge patterns between an overtrained symbol-hand association and a novel symbol hand association, responding in association with the movement of both hands, although the kinematics of the movement did not change between the two conditions. The activity change was not related to the pattern of the visual symbols, the movement kinematics, the monkeys' reaction times or the novelty of the visual symbols. The simple spike activity changed with throughout the learning process, but the concurrent complex spikes did not instruct that change. Although these neurons also have reward related activity, the reward-related and movement related signals were independent. We suggest that this mixed selectivity may facilitate the flexible learning of difficult reinforcement learning problems.

show abstract

“…This task was motivated by recent work using a similar simplified paradigm to investigate cerebellar-based incremental learning in non-human primates. 10,11 The second task, referred to throughout as the multiple . CC-BY-NC-ND 4.0 International license available under a (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity.…”

Section: Introductionmentioning

confidence: 99%

The role of the cerebellum in learning to predict reward: evidence from cerebellar ataxia

Nicholas

Amlang

Lin

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent findings in animals have challenged the traditional view of the cerebellum solely as the site of motor control, suggesting that the cerebellum may also be important for learning to predict reward from trial-and-error feedback. Yet, evidence for the role of the cerebellum in reward learning in humans is lacking. Moreover, open questions remain about which specific aspects of reward learning the cerebellum may contribute to. Here we address this gap through an investigation of multiple forms of reward learning in individuals with cerebellum dysfunction, represented by cerebellar ataxia cases. Nineteen participants with cerebellar ataxia and 57 age- and sex-matched healthy controls completed two separate tasks that required learning about reward contingencies from trial-and-error. To probe the selectivity of reward learning processes, the tasks differed in their underlying structure: while one task measured incremental reward learning ability alone, the other allowed participants to use an alternative learning strategy based on episodic memory alongside incremental reward learning. We found that individuals with cerebellar ataxia were profoundly impaired at reward learning from trial-and-error feedback on both tasks, but retained the ability to learn to predict reward based on episodic memory. These findings provide evidence from humans for a specific and necessary role for the cerebellum in incremental learning of reward associations based on reinforcement. More broadly, the findings suggest that alongside its role in motor learning, the cerebellum likely operates in concert with the basal ganglia to support reinforcement learning from reward.

show abstract

The mid-lateral cerebellum is necessary for reinforcement learning

Cited by 5 publications

References 31 publications

Mid-lateral cerebellar complex spikes encode multiple independent reward-related signals during reinforcement learning

Mid-lateral cerebellar complex spikes encode multiple independent reward-related signals during reinforcement learning

Mixed selectivity in the cerebellar Purkinje-cell response during visuomotor association learning

The role of the cerebellum in learning to predict reward: evidence from cerebellar ataxia

Contact Info

Product

Resources

About