The brain circuitry of saccadic eye movements, from brainstem to cortex, has been extensively studied during the last 30 years. The wealth of data gathered allowed the conception of numerous computational models. These models proposed descriptions of the putative mechanisms generating this data, and, in turn, made predictions and helped to plan new experiments. In this article, we review the computational models of the five main brain regions involved in saccade generation: reticular formation saccadic burst generators, superior colliculus, cerebellum, basal ganglia and premotor cortical areas. We present the various topics these models are concerned with: location of the feedback loop, multimodal saccades, long-term adaptation, on the fly trajectory correction, strategy and metrics selection, short-term spatial memory, transformations between retinocentric and craniocentric reference frames, sequence learning, to name the principle ones. Our objective is to provide a global view of the whole system. Indeed, narrowing too much the modelled areas while trying to explain too much data is a recurrent problem that should be avoided. Moreover, beyond the multiple research topics remaining to be solved locally, questions regarding the operation of the whole structure can now be addressed by building on the existing models.

Pilot studies revealed promising results regarding crushing virtual cigarettes to reduce tobacco addiction. In this study, 91 regular smokers were randomly assigned to two treatment conditions that differ only by the action performed in the virtual environment: crushing virtual cigarettes or grasping virtual balls. All participants also received minimal psychosocial support from nurses during each of 12 visits to the clinic. An affordable virtual reality system was used (eMagin HMD) with a virtual environment created by modifying a 3D game. Results revealed that crushing virtual cigarettes during 4 weekly sessions led to a statistically significant reduction in nicotine addiction (assessed with the Fagerström test), abstinence rate (confirmed with exhaled carbon monoxide), and drop-out rate from the 12-week psychosocial minimal-support treatment program. Increased retention in the program is discussed as a potential explanation for treatment success, and hypotheses are raised about self-efficacy, motivation, and learning.

Current learning theory provides a comprehensive description of how humans and other animals learn, and places behavioral flexibility and automaticity at heart of adaptive behaviors. However, the computations supporting the interactions between goal-directed and habitual decision-making systems are still poorly understood. Previous functional magnetic resonance imaging (fMRI) results suggest that the brain hosts complementary computations that may differentially support goal-directed and habitual processes in the form of a dynamical interplay rather than a serial recruitment of strategies. To better elucidate the computations underlying flexible behavior, we develop a dual-system computational model that can predict both performance (i.e., participants' choices) and modulations in reaction times during learning of a stimulus–response association task. The habitual system is modeled with a simple Q-Learning algorithm (QL). For the goal-directed system, we propose a new Bayesian Working Memory (BWM) model that searches for information in the history of previous trials in order to minimize Shannon entropy. We propose a model for QL and BWM coordination such that the expensive memory manipulation is under control of, among others, the level of convergence of the habitual learning. We test the ability of QL or BWM alone to explain human behavior, and compare them with the performance of model combinations, to highlight the need for such combinations to explain behavior. Two of the tested combination models are derived from the literature, and the latter being our new proposal. In conclusion, all subjects were better explained by model combinations, and the majority of them are explained by our new coordination proposal.

The basal ganglia nuclei form a complex network of nuclei often assumed to perform selection, yet their individual roles and how they influence each other is still largely unclear. In particular, the ties between the external and internal parts of the globus pallidus are paradoxical, as anatomical data suggest a potent inhibitory projection between them while electrophysiological recordings indicate that they have similar activities. Here we introduce a theoretical study that reconciles both views on the intra-pallidal projection, by providing a plausible characterization of the relationship between the external and internal globus pallidus. Specifically, we developed a mean-field model of the whole basal ganglia, whose parameterization is optimized to respect best a collection of numerous anatomical and electrophysiological data. We first obtained models respecting all our constraints, hence anatomical and electrophysiological data on the intrapallidal projection are globally consistent. This model furthermore predicts that both aforementioned views about the intra-pallidal projection may be reconciled when this projection is weakly inhibitory, thus making it possible to support similar neural activity in both nuclei and for the entire basal ganglia to select between actions. Second, we predicts that afferent projections are substantially unbalanced towards the external segment, as it receives the strongest excitation from STN and the weakest inhibition from the striatum. Finally, our study strongly suggest that the intrapallidal connection pattern is not focused but diffuse, as this latter pattern is more efficient for the overall selection performed in the basal ganglia.

In this article, we describe a new computational model of switching between path-planning and cue-guided navigation strategies. It is based on three main assumptions: (i) the strategies are mediated by separate memory systems that learn independently and in parallel; (ii) the learning algorithms are different in the two memory systems-the cueguided strategy uses a temporal-difference (TD) learning rule to approach a visible goal, whereas the path-planning strategy relies on a place-cell-based graph-search algorithm to learn the location of a hidden goal; (iii) a strategy selection mechanism uses TD-learning rule to choose the most successful strategy based on past experience. We propose a novel criterion for strategy selection based on the directions of goal-oriented movements suggested by the different strategies. We show that the selection criterion based on this "common currency" is capable of choosing the best among TD-learning and planning strategies and can be used to solve navigational tasks in continuous state and action spaces. The model has been successfully applied to reproduce rat behavior in two water-maze tasks in which the two strategies were Laurent Dollé, Denis Sheynikhovich-First authorship shared.

In a volatile environment where rewards are uncertain, successful performance requires a delicate balance between exploitation of the best option and exploration of alternative choices. It has theoretically been proposed that dopamine contributes to the control of this exploration-exploitation trade-off, specifically that the higher the level of tonic dopamine, the more exploitation is favored. We demonstrate here that there is a formal relationship between the rescaling of dopamine positive reward prediction errors and the exploration-exploitation trade-off in simple non-stationary multi-armed bandit tasks. We further show in rats performing such a task that systemically antagonizing dopamine receptors greatly increases the number of random choices without affecting learning capacities. Simulations and comparison of a set of different computational models (an extended Q-learning model, a directed exploration model, and a meta-learning model) fitted on each individual confirm that, independently of the model, decreasing dopaminergic activity does not affect learning rate but is equivalent to an increase in random exploration rate. This study shows that dopamine could adapt the exploration-exploitation trade-off in decision-making when facing changing environmental contingencies.

Action selection, the problem of choosing what to do next, is central to any autonomous agent architecture. We use here a multidisciplinary approach at the convergence of neuroscience, dynamical systems theory and autonomous robotics, in order to propose an efficient action selection mechanism based on a new model of the basal ganglia. We first describe new developments of contraction theory regarding locally projected dynamical systems. We exploit these results to design a stable computational model of the cortico-basothalamo-cortical loops. Based on recent anatomical data, we include usually neglected neural projections, which participate in performing accurate selection. Finally, the efficiency of this model as an autonomous robot action selection mechanism is assessed in a standard survival task. The model exhibits valuable dithering avoidance and energy-saving properPreprint submitted to Elsevier Science 18 May 2017 ties, when compared with a simple if-then-else decision rule.

Abstract. 200 words A biologically-inspired navigation system for the mobile rat-like robot named Psikharpax is presented, allowing for self-localization and autonomous navigation in an initially unknown environment. The ability of parts of the model (e.g. the strategy selection mechanism) to reproduce rat behavioral data in various maze tasks has been validated before in simulation. But the capacity of the model to work on a real robot platform had not been tested.This article presents our work on the implementation on the Psikharpax robot of two independent navigation strategies (a place-based planning strategy and a cueguided taxon strategy) and a strategy selection meta-controller. We show how our robot can memorize which was the optimal strategy in each situation, by means of a reinforcement learning algorithm. Moreover, a context detector enables the controller to quickly adapt to changes in the environment -recognized as new contexts -, and to restore previously acquired strategy preferences when a previously experienced context is recognized. This produces adaptivity closer to rat behavioral performances and constitutes a computational proposition of the role of the rat prefrontal cortex in strategy shifting. Moreover, such brain-inspired meta-controller may provide an advancement for learning architectures in robotics.A biologically inspired meta-control navigation system for the Psikharpax rat robot 2