The traveling salesrat: insights into the dynamics of efficient spatial navigation in the rodent

Abstract

Rodent spatial navigation requires the dynamic evaluation of multiple sources of information, including visual cues, self-motion signals and reward signals. The nature of the evaluation, its dynamics and the relative weighting of the multiple information streams are largely unknown and have generated many hypotheses in the field of robotics. We use the framework of the traveling salesperson problem (TSP) to study how this evaluation may be achieved. The TSP is a classical artificial intelligence NP-hard problem that requires an agent to visit a fixed set of locations once, minimizing the total distance traveled. We show that after a few trials, rats converge on a short route between rewarded food cups. We propose that this route emerges from a series of local decisions that are derived from weighing information embedded in the context of the task. We study the relative weighting of spatial and reward information and establish that, in the conditions of this experiment, when the contingencies are not in conflict, rats choose the spatial or reward optimal solution. There was a trend toward a preference for space when the contingencies were in conflict. We also show that the spatial decision about which cup to go to next is biased by the orientation of the animal. Reward contingencies are also shown to significantly and dynamically modulate the decision-making process. This paradigm will allow for further neurophysiological studies aimed at understanding the synergistic role of brain areas involved in planning, reward processing and spatial navigation. These insights will in turn suggest new neural-like architectures for the control of mobile autonomous robots.

Reservoir computing model of prefrontal cortex creates novel combinations of previous navigation sequences from hippocampal place-cell replay with spatial reward propagation

“…The model demonstrates the ability to accumulate and consolidate paths over multiple trials, and to exploit reverse replay. Here we examine these effects on the more extensive and variable dataset extracted from rat behavior [13]. We show the positive effects of replay on trajectories from rats trying to optimize spatial navigation in the TSP task.…”

“…The spatial resolution of trajectories are depicted at 20 points / m along the trajectory. Experiments were performed using navigation trajectories, including those displayed in Fig 1, based on data recorded from rats as they ran the TSP task [13] in a circular arena having a radius of 151cm. Twenty-one fixed feeders are distributed according to a spiral shape.…”

“…In a typical configuration, five feeders are baited with a food pellet. For a given configuration, the rat runs several trials which are initially random and inefficient, and become increasingly efficient over successive trials, characterizing the TSP behavior [13]. Rat data that characterizes the TSP behavior is detailed in S1 Text, section Rat navigation data.…”

“…We focus on the role of replay during the awake state, as the animal generates increasingly efficient trajectories between reward sites, across multiple trials. This trend toward near-optimal solutions is reminiscent of the classic Traveling Salesperson Problem (TSP) [13]. The TSP problem involves finding the shortest path that visits a set of “cities” on a 2D map.…”

“…It is a computationally complex problem, and is one of the most intensively studied problems in optimization [14, 15]. While it is clear that rats do not solve the TSP in the mathematical sense, they remarkably display a robust tendency towards such optimization [13]. It appears likely that such spatial navigation optimization involves planning and hence awake replay but the underlying neurophysiological mechanisms remain to be understood.…”