Neuro-inspired Navigation Strategies Shifting for Robots: Integration of a Multiple Landmark Taxon Strategy

Abstract

Abstract. Rodents have been widely studied for their adaptive navigation capabilities. They are able to exhibit multiple navigation strategies; some based on simple sensory-motor associations, while others rely on the construction of cognitive maps. We previously proposed a computational model of parallel learning processes during navigation which could reproduce in simulation a wide set of rat behavioral data and which could adaptively control a robot in a changing environment. In this previous robotic implementation the visual approach (or taxon) strategy was however paying attention to the intra-maze landmark only and learned to approach it. Here we replaced this mechanism by a more realistic one where the robot autonomously learns to select relevant landmarks. We show experimentally that the new taxon strategy is efficient, and that it combines robustly with the planning strategy, so as to choose the most efficient strategy given the available sensory information.

Design of a Control Architecture for Habit Learning in Robots

“…when there is perceptual aliasing [4], and can also require high computational costs and long time to propagate possible trajectories through internal representations [10]. We have previously shown that taking inspiration from the way rodents shift between different navigation strategies -a capacity which has been shown to be analogous to the shifts between goaldirected and habitual decision systems [14] -can be applied to a robotic platform to enable to automatically exploit the advantages of each strategy [4,3]. However, these experiments only involved navigation behaviors from one location to another.…”

Skill Learning by Autonomous Robotic Playing Using Active Learning and Exploratory Behavior Composition

“…The switching is done randomly [18], or by either majority vote, rank vote, Boltzmann Multiplication or Boltzmann Addition [19]. Similar work has been done in a navigation task by Caluwaerts et al [20], [21]. Their biologically inspired approach uses three different experts, namely a taxon expert (model-free), a planning expert (modelbased), and an exploration expert, i.e.…”

“…Our work combines a model-free playing system and a model-based behavior generation system. Work on switching between model-free and model-based controllers was proposed in many areas of robotics (e.g., by Daw et al, 2005 ; Dollé et al, 2010 ; Keramati et al, 2011 ; Caluwaerts et al, 2012a , b ; Renaudo et al, 2014 , 2015 ). The selection of different controllers is typically done by measuring the uncertainty of the controller's predictions.…”

Adaptive Coordination of Multiple Learning Strategies in Brains and Robots

“…In addition, the robot could use a model-free reinforcement learning strategy to learn movements in 8 cardinal directions in response to perceived salient features within the environment (i.e., stimuli in the vocabulary of psychology). The latter MF RL component of the model was later improved in [57] to make it able to learn movements away from visual features when needed. The proposed algorithm for the online coordination of MB and MF RL was based on the computational neuroscience model of Dollé and colleagues [58,59,9], which has been presented in the previous section and sketched in Fig.…”

“…Nevertheless, some limitations and perspectives of this seminal work ought to be mentioned here. First, the coordination component of the algorithm (which is called the meta-controller in [9,46,57]) slowly learns through MF RL (in addition to the MF RL mechanism used within the MF system dedicated to the MF strategy) which strategy is the most appropriate in each part of the environment (In other words, the model involves a hierarchical learning process in addition to the parallel learning process between MB and MF strategies). While this is good for the robot to be able to memorize specific coordination patterns for each context (i.e., for each configuration of the goal location within the arena), this nevertheless requires a long time to achieve a good coordination within each context.…”