Clone-structured graph representations enable flexible learning and vicarious evaluation of cognitive maps

George, Dileep; Rikhye, Rajeev; Gothoskar, Nishad; Guntupalli, J. Swaroop; Dedieu, Antoine; Lázaro-Gredilla, Miguel

doi:10.1038/s41467-021-22559-5

Cited by 57 publications

(103 citation statements)

References 67 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Section: Box 2: Building Latent State Representations From Sequencesmentioning

confidence: 99%

“…The clone-structured cognitive graph (CSCG) model 62 is an elegant approach for building de-aliased state-spaces. Here, hippocampus contains multiple 'clone' cells for each sensory observation 62,63 . Now, one hippocampal 'frog' clone cell responds to a frog in one location, and another responds if a frog appears elsewhere (Figure 3).…”

Section: Box 2: Building Latent State Representations From Sequencesmentioning

confidence: 99%

“…We have seen models that build latent state representations, and models that path integrate. If these principles could be combined, we could build a powerful system that learns arbitrary latent states from sensory observations (like CSCG 62 ) but additionally generalises these representations (like path integration models 78,85 ) and composes them arbitrarily. For abstract representations to be reused (generalised) in different sensory environments, the same abstract locations must be 'linked' to different sensory observations.…”

Section: Box 4: Generalising With Memoriesmentioning

confidence: 99%

See 2 more Smart Citations

How to build a cognitive map: insights from models of the hippocampal formation

Whittington¹,

McCaffary²,

Bakermans³

et al. 2022

Preprint

View full text Add to dashboard Cite

Learning and interpreting the structure of the environment is an innate feature of biological systems, and is integral to guiding flexible behaviours for evolutionary viability. The concept of a cognitive map has emerged as one of the leading metaphors for these capacities, and unravelling the learning and neural representation of such a map has become a central focus of neuroscience. While experimentalists are providing a detailed picture of the neural substrate of cognitive maps in hippocampus and beyond, theorists have been busy building models to bridge the divide between neurons, computation, and behaviour. These models can account for a variety of known representations and neural phenomena, but often provide a differing understanding of not only the underlying principles of cognitive maps, but also the respective roles of hippocampus and cortex. In this Perspective, we bring many of these models into a common language, distil their underlying principles of constructing cognitive maps, provide novel (re)interpretations for neural phenomena, suggest how the principles can be extended to account for prefrontal cortex representations and, finally, speculate on the role of cognitive maps in higher cognitive capacities.

show abstract

Section: Box 2: Building Latent State Representations From Sequencesmentioning

confidence: 99%

Section: Box 2: Building Latent State Representations From Sequencesmentioning

confidence: 99%

Section: Box 4: Generalising With Memoriesmentioning

confidence: 99%

See 1 more Smart Citation

How to build a cognitive map: insights from models of the hippocampal formation

Whittington¹,

McCaffary²,

Bakermans³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In particular, we identify head-direction cells, border cells with both ego and allocentric responses, and place cells. Critically, unlike other recent models of spatial representations 25, 26 , we do so without the need for any allocentric inputs, using only information that is plausibly available to the brain. Thus, our network links this diverse array of cells to a simple predictive framework, it poses hypotheses about the circuit dynamics that produce these responses, and embodies a model linking sensory information, neural activity, and spatial behaviour.…”

Section: Introductionmentioning

confidence: 99%

A model of egocentric to allocentric understanding in mammalian brains

Uría

Ibarz

Banino

et al. 2020

Preprint

View full text Add to dashboard Cite

In the mammalian brain, allocentric representations support efficient self-location and flexible navigation. A number of distinct populations of these spatial responses have been identified but no unified function has been shown to account for their emergence. Here we developed a network, trained with a simple predictive objective, that was capable of mapping egocentric information into an allocentric spatial reference frame. The prediction of visual inputs was sufficient to drive the appearance of spatial representations resembling those observed in rodents: head direction, boundary vector, and place cells, along with the recently discovered egocentric boundary cells, suggesting predictive coding as a principle for their emergence in animals. The network learned a solution for head direction tracking convergent with known biological connectivity, while suggesting a possible mechanism of boundary cell remapping. Moreover, like mammalian representations, responses were robust to environmental manipulations, including exposure to novel settings, and could be replayed in the absence of perceptual input, providing the means for offline learning. In contrast to existing reinforcement learning approaches, agents equipped with this network were able to flexibly reuse learnt behaviours - adapting rapidly to unfamiliar environments. Thus, our results indicate that these representations, derived from a simple egocentric predictive framework, form an efficient basis-set for cognitive mapping.

show abstract

“…Thus, it remains unclear how sleep impacts the learning of network structures. It has recently been argued that the hippocampus and parahippocampal structures which support spatial memory and navigation may have evolved in humans to support the learning of knowledge networks more broadly (Behrens et al, 2018;Bellmund, Gardenfors, Moser, & Doeller, 2018;Epstein, Patai, Julian, & Spiers, 2017;George et al, 2021;Spiers, 2020;Whittington et al, 2020). This has extended to concepts in reinforcement learning where optimal policies for learning new information need to be developed (Stachenfeld, Botvinick, & Gershman, 2017).…”

Section: Introductionmentioning

confidence: 99%

Learning graph networks: sleep targets highly connected global and local nodes for consolidation

Feld

Bernard

Rawson

et al. 2021

Preprint

View full text Add to dashboard Cite

Much of our long-term knowledge is organised in complex networks. Sleep is thought to be critical for abstracting knowledge and enhancing important item memory for long-term retention. Thus, sleep should aid the development of memory for networks and the abstraction of their structure for efficient storage. However, this remains unknown because past sleep studies have focused on discrete items. Here we explored the impact of sleep (night-sleep/day-wake within-subject paradigm) on memory for graph-networks where some items were important due to dense local connections (degree centrality) or, independently, important due to greater global connections (closeness / betweenness centrality). A network of 27 planets (nodes) sparsely interconnected by 36 teleporters (edges) was learned via discrete associations without explicit indication of any network structure. Despite equivalent exposure to all connections in the network, we found that memory for the links between items with high local centrality or high global centrality were better retained after sleep. These results highlight that sleep has the capacity for strengthening both global and local structure from the world and abstracting over multiple experiences to efficiently form internal networks of knowledge.

show abstract

Clone-structured graph representations enable flexible learning and vicarious evaluation of cognitive maps

Cited by 57 publications

References 67 publications

How to build a cognitive map: insights from models of the hippocampal formation

How to build a cognitive map: insights from models of the hippocampal formation

A model of egocentric to allocentric understanding in mammalian brains

Learning graph networks: sleep targets highly connected global and local nodes for consolidation

Contact Info

Product

Resources

About