A Neurocomputational Model of Goal-Directed Navigation in Insect-Inspired Artificial Agents

Abstract

Despite their small size, insect brains are able to produce robust and efficient navigation in complex environments. Specifically in social insects, such as ants and bees, these navigational capabilities are guided by orientation directing vectors generated by a process called path integration. During this process, they integrate compass and odometric cues to estimate their current location as a vector, called the home vector for guiding them back home on a straight path. They further acquire and retrieve path integration-based vector memories globally to the nest or based on visual landmarks. Although existing computational models reproduced similar behaviors, a neurocomputational model of vector navigation including the acquisition of vector representations has not been described before. Here we present a model of neural mechanisms in a modular closed-loop control—enabling vector navigation in artificial agents. The model consists of a path integration mechanism, reward-modulated global learning, random search, and action selection. The path integration mechanism integrates compass and odometric cues to compute a vectorial representation of the agent's current location as neural activity patterns in circular arrays. A reward-modulated learning rule enables the acquisition of vector memories by associating the local food reward with the path integration state. A motor output is computed based on the combination of vector memories and random exploration. In simulation, we show that the neural mechanisms enable robust homing and localization, even in the presence of external sensory noise. The proposed learning rules lead to goal-directed navigation and route formation performed under realistic conditions. Consequently, we provide a novel approach for vector learning and navigation in a simulated, situated agent linking behavioral observations to their possible underlying neural substrates.

Merging information in the entorhinal cortex: what can we learn from robotics experiments and modeling?

“…Starting from homing behaviors and detour problems in navigation , we conclude as others (Spiers and Gilbert, 2015) that 'path integration' or 'dead reckoning' (Etienne and Jeffery, 2004) may (Worsley et al, 2001;Wolbers et al, 2007;Stangl et al, 2018) or may not (Shrager et al, 2008;Kim et al, 2013) involve hippocampal-parahippocampal structures, and that this involvement perhaps depends on the distance navigated (Gaussier et al, 2007;Arnold et al, 2014). Our different studies, as well as work on insect modeling (Stone et al, 2017;Schwarz et al, 2017;Goldschmidt et al, 2017), support the hypothesis that PI can be obtained without the need of EC grid cells. HD cells or any kind of internal compass associated with local memories (to support temporal integration) can be used to perform PI in different brain structures [different kinds of reset/preset and different time constants are sufficient to explain many different cells found in the RSC and the parietal cortex (P.G.…”

Principles of Insect Path Integration

“…Comparable estimates are found using the median radius of search initiation points [37]. It is unclear what could cause such large path integration errors, or whether current neurocomputational models can cope with them [64,77]. As even unrealistically large sensory errors cannot come close to such uncertainty (Box 1(IV)), path integration error may be largely due to internal path integration update noise rather than input/output noise.…”

“…Similarly, it is unclear whether search is an emergent property of the path integration system itself [59,64,120] or requires additional neural control. A reward-based motivational system may be able to orchestrate largely independent functional modules, of which path integration, search, route following, and landmark navigation are examples [77,121]. If so, path integration can be understood independently of the rest of an insect’s navigational system.…”

The internal maps of insects

“…Insect navigation has been explored in a wide range of mathematical and computational models (e.g. Arena et al, 2013;Baddeley et al, 2012;Cartwright and Collett, 1983;Cruse and Wehner, 2011;Dewar et al, 2014;Goldschmidt et al, 2017;Hartmann and Wehner, 1995;Mathews et al, 2009;Möller and Vardy, 2006;Vardy and Möller, 2005;Wittmann and Schwegler, 1995), with a number of these also demonstrated in robots (e.g. Kodzhabashev and Mangan, 2015;Lambrinos et al, 2000;Mathews et al, 2010;Möller, 2000;Smith et al, 2007).…”

“…Moreover, as demonstrated in several modelling studies (e.g. Cruse and Wehner, 2011;Goldschmidt et al, 2017), combining the current home vector state with a vector memory allows insects to take novel shortcuts between their current location and the vector memory location. For example, a bee reaching an empty feeder may take a (novel) flight directly towards an alternative (known) feeder location (Menzel et al, 2011); and an ant forced to make a detour on an outward journey to a feeder will take the (novel) direct path from the end of the detour to the feeder (Collett et al, 1999).…”