Classical conditioning drives learned reward prediction signals in climbing fibers across the lateral cerebellum

Abstract

27Classical models of cerebellar learning posit that climbing fibers operate according to a 28 supervised learning rule to instruct changes in motor output by signaling the occurrence of 29 movement errors. However, cerebellar output is also associated with non-motor behaviors, and 30 recently with modulating reward association pathways in the VTA. To test how the cerebellum 31 processes reward related signals in the same type of classical conditioning behavior typically 32 studied to evaluate reward processing in the VTA and striatum, we have used calcium imaging to 33 visualize instructional signals carried by climbing fibers across the lateral cerebellum before and 34 after learning. We find distinct climbing fiber responses in three lateral cerebellar regions that can 35 each signal reward prediction. These instructional signals are well suited to guide cerebellar 36 learning based on reward expectation and enable a cerebellar contribution to reward driven 37 behaviors, thus suggesting a broad role for the lateral cerebellum in reward-based learning. 38 39 Introduction 40The cerebellum plays a key role in motor control and motor learning. However, it is becoming 41 widely appreciated that the cerebellum also contributes significantly to many non-motor processes 42 (Schmahmann, 1991) such as cognition (Kim et al., 1994), social processing (Van Overwalle et 43 al., 2014), aggression (Reis et al., 1973 and emotion (Schmahmann and Caplan, 2006). 44Moreover, cerebellar deficits are associated with non-motor conditions including autism spectrum 45 disorders (Wang et al., 2014), deficits of language processing and vocal learning (Ackermann, 46 2008), schizophrenia (Mothersill et al., 2016 and other impairments (Ivry and Spencer, 2004). 47Together, these observations strongly suggest that the cerebellum, and cerebellar learning, must 48 operate in a manner compatible with adjusting motor as well as non-motor processes. 49Classical models posit that cerebellar learning is instructed by signals carried by an afferent fiber 50 projection from the inferior olive called climbing fibers (Albus, 1971; Ito, 1972; Marr, 1969). Such 51 models suggest that climbing fibers operate according to a supervised learning rule to instruct 52 changes in motor output by signaling the occurrence of movement errors. However, it is unclear 53 how such a supervised learning rule could contribute to modification of many non-motor 54 behaviors. In line with this view, accumulating evidence has demonstrated a wide range of 55 climbing fiber responses that are distinct from motor error signals (Kitazawa et al., 1998; Streng 56 et al., 2017), including prediction error and reward related signals (Heffley et al., 2018; Kostadinov 57 et al., 2019; Larry et al., 2019;Ohmae and Medina, 2015). Together, such data provide evidence 58 that the cerebellum may utilize alternative learning rules to support a wider range of behaviors 59 than can be driven by error-based supervised learning alone. 60Under what conditions might the climbing fiber...

Cerebellar Activation Bidirectionally Regulates Nucleus Accumbens Core and Medial Shell

“…Further support for non-motor CB roles stems from clinical translational studies, which have linked CB dysfunction with neurodevelopmental disorders, posttraumatic stress disorder, generalized anxiety disorder, addiction, and cognitive and emotional disturbances known as cerebellar cognitive affective syndrome 8,[10][11][12][13][14][15][16][17][18] . These findings are further corroborated by evidence from animal studies, which solidify a role for the CB in the processing of valence, reward, reward anticipation and omission [19][20][21][22][23] ; emotional learning and aggression [24][25][26][27][28][29][30] ; and motivation [31][32][33] .…”

Population coding in the cerebellum: a machine learning perspective

“…The probability of a complex spike depends on many factors beyond the presence or absence of a prediction error. For example, while sudden occurrence of a stimulus by itself may not produce a complex spike in the P-cell of a naïve animal, it will do so if the animal has learned to associate that stimulus with an aversive event (Ohmae and Medina, 2015), with a greater reward (Heffley and Hall, 2019;Kostadinov et al, 2019;Larry et al, 2019), or a previous performance error (Junker et al, 2018). Withholding of reward when it was expected can also modulate complex spike rates (Heffley et al, 2018;Kostadinov et al, 2019).…”

“…In summary, several recent reports have shown that complex spikes carry information about the reward value associated with a stimulus (Heffley and Hall, 2019;Kostadinov et al, 2019;Larry et al, 2019). To reconcile this fact with the idea that complex spikes carry an error signal, we can consider an olivary neuron that receives two inputs: inhibition from a DCN neuron, and excitation from a neuron in the superior colliculus.…”

Complex spike firing adapts to saliency of inputs and engages readiness to act

“…An important difference with paradigms generally described in other works (e.g. (Heffley and Hull, 2019;Heffley et al, 2018;Kostadinov et al, 2019;Larry et al, 2019;Tsutsumi et al, 2019)) is that the water delivery was triggered by the first lick of bout, in the moment when the laser beam in front of the lick-port was interrupted; as a result, mice had to lick before the reward was given, so that they could not use the presence of water nor the valve click as a cue. During no-go trials, mice were not supposed to lick and licking was consequently not rewarded.…”

“…The mice had to lick first before they received a reward. The first lick was therefore unrewarded and, given the potential importance of complex spikes for reward expectation (Heffley and Hull, 2019;Heffley et al, 2018;Kostadinov et al, 2019;Tsutsumi et al, 2019), we repeated this analysis while aligning the complex spikes with the second lick, thus the timing of the reward. This did not reveal a clear coupling between the timing of complex spike firing and reward delivery (Fig.…”