Functional Relevance of Different Basal Ganglia Pathways Investigated in a Spiking Model with Reward Dependent Plasticity

Abstract

The brain enables animals to behaviorally adapt in order to survive in a complex and dynamic environment, but how reward-oriented behaviors are achieved and computed by its underlying neural circuitry is an open question. To address this concern, we have developed a spiking model of the basal ganglia (BG) that learns to dis-inhibit the action leading to a reward despite ongoing changes in the reward schedule. The architecture of the network features the two pathways commonly described in BG, the direct (denoted D1) and the indirect (denoted D2) pathway, as well as a loop involving striatum and the dopaminergic system. The activity of these dopaminergic neurons conveys the reward prediction error (RPE), which determines the magnitude of synaptic plasticity within the different pathways. All plastic connections implement a versatile four-factor learning rule derived from Bayesian inference that depends upon pre- and post-synaptic activity, receptor type, and dopamine level. Synaptic weight updates occur in the D1 or D2 pathways depending on the sign of the RPE, and an efference copy informs upstream nuclei about the action selected. We demonstrate successful performance of the system in a multiple-choice learning task with a transiently changing reward schedule. We simulate lesioning of the various pathways and show that a condition without the D2 pathway fares worse than one without D1. Additionally, we simulate the degeneration observed in Parkinson's disease (PD) by decreasing the number of dopaminergic neurons during learning. The results suggest that the D1 pathway impairment in PD might have been overlooked. Furthermore, an analysis of the alterations in the synaptic weights shows that using the absolute reward value instead of the RPE leads to a larger change in D1.

Untangling Basal Ganglia Network Dynamics and Function: Role of Dopamine Depletion and Inhibition Investigated in a Spiking Network Model

“…The inserted connectivity, represented by activation of specific synapses between cortex and striatum, was assumed to have been learned from previous behavioral experiences. For models tackling this problem, see Potjans et al (2011), Stewart et al (2012), and Berthet et al (2016). Strictly speaking, we tested with our model how suited BG are for transferring learned state action signals to motor-related output nuclei in thalamus or brainstem.…”

“…It was assumed that the connectivity between cortex and striatum had been learned such that cortical state 1 (S1) activated a MSN D1 pool that in turn inhibited action 1 SNr neurons and at the same time activated MSN D2 neurons that increased the activity of action 2 SNr neurons (i.e., non–action 1 SNr neurons) through the indirect pathway. See Berthet et al (2016) for how such connectivity can be learned. The opposite connectivity was assumed for state 2 when activating MSN D1 and D2 neurons.…”

Roles for globus pallidus externa revealed in a computational model of action selection in the basal ganglia

“…The basal ganglia are an evolutionarily conserved group of subcortical nuclei, which have long been implicated in action selection (Redgrave et al, 1999;Hikosaka et al, 2000;Frank et al, 2004;Frank, 2005;Schroll et al, 2012;Lindahl et al, 2013;Grillner and Robertson, 2016;Stephenson-Jones et al, 2011). Several computational models have been developed, examining their role in action selection (Mink, 1996;Hikosaka et al, 2000;Gurney et al, 2001a,b;Frank et al, 2004;Schroll et al, 2012;Kamali Sarvestani et al, 2011;Berthet et al, 2016). They propose the basal ganglia as a 'selection machine' resolving conflicts between competing behaviours for common and restricted motor resources (Redgrave et al, 1999;Schroll and Hamker, 2013;Frank, 2005).…”

Animal models in psychiatric research: The RDoC system as a new framework for endophenotype-oriented translational neuroscience

“…Three reference points were suggested to specify the network, responsible for 1 of the behaviour characteristics selected by Pankseep: (1) similar neuronal circuits maintain coherent functions; (2) artificial stimulation of the particular network (with a pharmacological, electrophysiological, and opto-/chemo-genetic means) generates predicted responses; and (3) changes in the carriers of the network activity (neurotransmitters and other substances with messenger activity) predict the behavioural changes (Panksepp, 1998). A lot of research aims to decipher the circuitries that predominantly control (1) arousal and sleep (Herrera et al., 2016, Kim et al., 2012, Landgraf et al., 2016), (2) fear, (3) anxiety, (4) aversive memories (Bravo-Rivera et al., 2014, Kim et al., 2013, McCullough et al., 2016, Tovote et al., 2015), (5) reward (Kelley, 2004, Kelley and Berridge, 2002, Smith et al., 2011), (6) attention and motivation (Berthet et al., 2016, Carli and Invernizzi, 2014, Kim, 2013), (7) goal-oriented behaviour and habits (Burguière et al., 2015, Chersi et al., 2013, Frank, 2011, Gremel and Costa, 2013, Medendorp et al., 2011) and (8) social functions (Konopka and Roberts, 2016, Kragel et al., 2015, Sladky et al., 2015, Zikopoulos and Barbas, 2013). …”