Coarse-Grained Neural Network Model of the Basal Ganglia to Simulate Reinforcement Learning Tasks

Abstract

Computational models of the basal ganglia (BG) provide a mechanistic account of different phenomena observed during reinforcement learning tasks performed by healthy individuals, as well as by patients with various nervous or mental disorders. The aim of the present work was to develop a BG model that could represent a good compromise between simplicity and completeness. Based on more complex (fine-grained neural network, FGNN) models, we developed a new (coarse-grained neural network, CGNN) model by replacing layers of neurons with single nodes that represent the collective behavior of a given layer while preserving the fundamental anatomical structures of BG. We then compared the functionality of both the FGNN and CGNN models with respect to several reinforcement learning tasks that are based on BG circuitry, such as the Probabilistic Selection Task, Probabilistic Reversal Learning Task and Instructed Probabilistic Selection Task. We showed that CGNN still has a functionality that mirrors the behavior of the most often used reinforcement learning tasks in human studies. The simplification of the CGNN model reduces its flexibility but improves the readability of the signal flow in comparison to more detailed FGNN models and, thus, can help to a greater extent in the translation between clinical neuroscience and computational modeling.

The onset of motor learning impairments in Parkinson’s disease: a computational investigation

“…That is, ultimately only one of Go or NoGo pathways activity predominates in the excitation/inhibition of the GPi/thalamus, thus amplifying or decreasing the force of movement. As in other studies [2,5,20], here we adopt the distinction between two sub-populations of neurons in the striatum, based on differences in biochemistry and efferent projections.…”

“…The phasic signal is fast-timescale and spans milliseconds, whereas the tonic signal is slow-timescale and can span minutes or hours. Phasic changes in DA play a key role in synaptic plasticity and reinforcement learning, as are thought to occur during error feedback (e.g., [2,5,20,27]), causing the two subpopulations of striatal neurons (direct and indirect) to independently learn positive and negative reinforcement values of responses. In particular, phasic increases in DA, due to positive feedback, result in increased activity in the direct pathway and suppression of the indirect pathway.…”

“…Its primary function is that of controlling and regulating the activities of the motor and premotor cortical areas for executing smooth movements. Various authors [1][2][3][4][5][6] agree on the role of BG in selectively facilitating the execution of a single elementary motor command while suppressing all competing others. Current theories of motor learning claim that early learning is controlled by dopamine-modulated plasticity in the BG, which trains parallel cortical pathways through unsupervised plasticity at corticostriatal synapses [7,8].…”