Mid-lateral cerebellar complex spikes encode multiple independent reward-related signals during reinforcement learning

Abstract

Although the cerebellum has been implicated in simple reward-based learning recently, the role of complex spikes (CS) and simple spikes (SS), their interaction and their relationship to complex reinforcement learning and decision making is still unclear. Here we show that in a context where a non-human primate learned to make novel visuomotor associations, classifying CS responses based on their SS properties revealed distinct cell-type specific encoding of the probability of failure after the stimulus onset and the non-human primate’s decision. In a different context, CS from the same cerebellar area also responded in a cell-type and learning independent manner to the stimulus that signaled the beginning of the trial. Both types of CS signals were independent of changes in any motor kinematics and were unlikely to instruct the concurrent SS activity through an error based mechanism, suggesting the presence of context dependent, flexible, multiple independent channels of neural encoding by CS and SS. This diversity in neural information encoding in the mid-lateral cerebellum, depending on the context and learning state, is well suited to promote exploration and acquisition of wide range of cognitive behaviors that entail flexible stimulus-action-reward relationships but not necessarily motor learning.

The role of the cerebellum in learning to predict reward: evidence from cerebellar ataxia

“…[8][9][10][11][12][13][14][15][16][17][18][19] This work has suggested that, alongside its role in motor learning, the cerebellum likely operates in concert with the basal ganglia to support reinforcement learning from reward. Our study corroborates these findings from animal models, [10][11][12][13][14][15][16][17][18][19] providing evidence that the human cerebellum is necessary for learning associations from reward. In comparison to age-and sex-matched healthy controls, CA participants were impaired at reward-based learning from trial-and-error.…”

“…[5][6][7] However, recent findings have challenged the notion that the cerebellum is solely responsible for supervised learning of motor behavior and instead suggest that the cerebellum may also be involved in the processing of reward more generally. [8][9][10][11][12][13][14][15][16][17][18][19] In particular, climbing fiber inputs to the cerebellum encode expected reward, 13,15,17,19 and cerebellar Purkinje cells have been found to report reward-based prediction errors. 11,12,18 These signals are essential ingredients for reinforcement learning, or learning that allows an organism to determine from trial-and-error feedback which actions should be taken in order to maximize future expected reward.…”

“…[8][9][10][11][12][13][14][15][16][17][18][19] In particular, climbing fiber inputs to the cerebellum encode expected reward, 13,15,17,19 and cerebellar Purkinje cells have been found to report reward-based prediction errors. 11,12,18 These signals are essential ingredients for reinforcement learning, or learning that allows an organism to determine from trial-and-error feedback which actions should be taken in order to maximize future expected reward. The presence of reward-related processing in the cerebellum suggests that it may play a role in reinforcement learning alongside its capacity for supervised motor learning.…”

“…20 This proposal challenges not only our current understanding of cerebellar function, but also our understanding of how the brain learns from reward more broadly. 5,21 Although research on the cerebellum's function in reward learning is growing, the vast majority of work has been done in animal models, [10][11][12][13][14][15][16][17][18][19] and evidence in humans remains limited. Human Running title: REWARD LEARNING IN THE CEREBELLUM 4 neuroimaging studies have revealed correlational evidence that the cerebellum is involved in tasks unrelated to movement, 22 however, despite some reports of BOLD activity in the cerebellum in response to reward across several early imaging studies, [23][24][25] more direct investigations of the role of cerebellum in reward-related behaviors in humans are lacking.…”

Activation of Cerebellar Lobule VI PCTH+–Med Pathway Selectively blocks METH-induced Conditioned Place Preference in mice

“…Rodent studies showed cocaine-paired olfactory cue increased activity of the granule cell layer of the cerebellar vermis (Carbo-Gas, Vazquez-Sanroman, Aguirre-Manzo, et al, 2014; Carbo-Gas, Vazquez-Sanroman, Gil-Miravet, et al, 2014), and a random cocaine-odour pairing procedure triggered higher cFos expression in the Med (Carbo-Gas, Vazquez-Sanroman, Gil-Miravet, et al, 2014). Non-human primate PCs encoded multiple independent reward-related signals during reinforcement learning (Sendhilnathan, Ipata, & Goldberg, 2021). Here, we found Med CaMKII neurons were notably excited during CPP training, but not CPP test, suggesting Med glutamatergic neurons are responsive to METH instead of METH-paired conditioned stimulus.…”