Interactions of spatial strategies producing generalization gradient and blocking: A computational approach

Abstract

We present a computational model of spatial navigation comprising different learning mechanisms in mammals, i.e., associative, cognitive mapping and parallel systems. This model is able to reproduce a large number of experimental results in different variants of the Morris water maze task, including standard associative phenomena (spatial generalization gradient and blocking), as well as navigation based on cognitive mapping. Furthermore, we show that competitive and cooperative patterns between different navigation strategies in the model allow to explain previous apparently contradictory results supporting either associative or cognitive mechanisms for spatial learning. The key computational mechanism to reconcile experimental results showing different influences of distal and proximal cues on the behavior, different learning times, and different abilities of individuals to alternatively perform spatial and response strategies, relies in the dynamic coordination of navigation strategies, whose performance is evaluated online with a common currency through a modular approach. We provide a set of concrete experimental predictions to further test the computational model. Overall, this computational work sheds new light on inter-individual differences in navigation learning, and provides a formal and mechanistic approach to test various theories of spatial cognition in mammals.

A general model of hippocampal and dorsal striatal learning and decision making

“…Similarly to earlier work in spatial RL (15,(47)(48)(49), the two systems in our model estimate value using qualitatively different strategies, which can cause them to generate divergent predictions for the optimal policy. The dorsal striatal component uses an MF temporal difference (TD) method (50) to learn stimulusresponse associations directly from egocentric sensory inputs given by landmark cells (LCs) tuned to landmarks at given distances and egocentric directions from the agent (Fig.…”

Modeling awake hippocampal reactivations with model-based bidirectional search

“…While the two may be difficult to disentangle experimentally, the present work highlights some key properties of model-based reactivations, such as the ability to generate imaginary sequences, which goes beyond model-free replay of past experience. From a broader perspective, the detailed offline reactivation method proposed here could constitute a refinement of the model-based component of architectures that combine model-based and model-free reinforcement learning (Dollé et al, 2010;Caluwaerts et al, 2012;Renaudo et al, 2014), and which can account for a wider range of animal complex spatial navigation behaviors (Dollé et al, 2008(Dollé et al, , 2018.…”

When Artificial Intelligence and Computational Neuroscience Meet

“…Finally, in the last decade computational neuroscientists have discovered that the classical distinction between learning strategies considered in AI and Machine Learning, namely model-based and model-free RL, also applies to Neuroscience by capturing different experimentally observed behavioral strategies in mammals, and related brain activities in different networks involving different basal ganglia territories [Daw et al, 2005;Khamassi and Humphries, 2012;Dollé et al, 2018]. More precisely, it turns out that mammals often start learning a task by trying to build an internal model of the task states, actions and transitions between them.…”

“…For instance, Daw and colleagues have suggested that the relative uncertainty within each learning system could help the brain decide which system should control behavior at any given moment [Daw et al, 2005]. In addition, more recent models can explain a variety of animal behavior in different tasks by employing a meta-controller which meta-learns which learning system was the most efficient in each state of each task [Dollé et al, 2018]. This suggests principles for the coordination of multiple learning systems which could inspire Artificial Intelligence in return.…”