Path planning versus cue responding: a bio-inspired model of switching between navigation strategies

Abstract

In this article, we describe a new computational model of switching between path-planning and cue-guided navigation strategies. It is based on three main assumptions: (i) the strategies are mediated by separate memory systems that learn independently and in parallel; (ii) the learning algorithms are different in the two memory systems-the cueguided strategy uses a temporal-difference (TD) learning rule to approach a visible goal, whereas the path-planning strategy relies on a place-cell-based graph-search algorithm to learn the location of a hidden goal; (iii) a strategy selection mechanism uses TD-learning rule to choose the most successful strategy based on past experience. We propose a novel criterion for strategy selection based on the directions of goal-oriented movements suggested by the different strategies. We show that the selection criterion based on this "common currency" is capable of choosing the best among TD-learning and planning strategies and can be used to solve navigational tasks in continuous state and action spaces. The model has been successfully applied to reproduce rat behavior in two water-maze tasks in which the two strategies were Laurent Dollé, Denis Sheynikhovich-First authorship shared.

Neuro-inspired Navigation Strategies Shifting for Robots: Integration of a Multiple Landmark Taxon Strategy

“…The model selects among two parallelly learned navigation strategies: a response strategy learning to orient towards relevant cues in the visual field; a place strategy building a map of place cells and planning trajectories between different locations in the arena. This model constitutes an extension to a previously published model of multiple navigation strategies [17] which was Relative proportion of selection of the planning strategy over the taxon strategy during the second part of the full model experiment, averaged over 50cm×50cm sliding windows. Note the 0 to 60% scale.…”

“…The planning expert (place cells + planning graph) has been extensively described in [17,18]. In following sections, we provide the equations for the newly implemented taxon expert and for its coordination with other experts by the gating network.…”

“…A specificity of the [17] model is that the adaptive navigation experts (here taxon and planning) receive reward signals even when they did not generate the movement, so that they can learn from each other: for the taxon, the Qvalue of the landmark whose direction is the closest to the direction of the real movement (no matter if it was generated by the taxon itself, by the planning or by the exploration expert) is updated; for the planning the position of the reward is recorded whichever expert led the robot there.…”

“…In previous work, we proposed a modular computational model that can explain a wide range of behavioral data recorded in rodent laboratory maze tasks [17]. We implemented the model in a robot and showed that it enabled the robot to benefit from the particular advantages of each strategy (the planning strategy being efficient far from the goal, the taxon strategy being more precise in adjusting the robot's trajectory near the goal based on vision) by autonomously learning which strategy was the most efficient in each part of the environment [18].…”

A biologically inspired meta-control navigation system for the Psikharpax rat robot

“…), coordinated by a meta-controller for strategy-shifting which has been previously shown to better reproduce rodents' behavioral performance than single navigation strategies [33]. Thus we extracted the principles of each previously studied components of rats' currently known cognitive architecture for navigation: place cells, path integration component, path planner, reinforcement learner.…”