Rapid learning of predictive maps with STDP and theta phase precession

Abstract

The predictive map hypothesis is a promising candidate principle for hippocampal function. A favoured formalisation of this hypothesis, called the successor representation, proposes that each place cell encodes the expected state occupancy of its target location in the near future. This predictive framework is supported by behavioural as well as electrophysiological evidence and has desirable consequences for both the generalisability and efficiency of reinforcement learning algorithms. However, it is unclear how the successor representation might be learnt in the brain. Error-driven temporal difference learning, commonly used to learn successor representations in artificial agents, is not known to be implemented in hippocampal networks. Instead, we demonstrate that spike-timing dependent plasticity (STDP), a form of Hebbian learning, acting on temporally compressed trajectories known as “theta sweeps”, is sufficient to rapidly learn a close approximation to the successor representation. The model is biologically plausible – it uses spiking neurons modulated by theta-band oscillations, diffuse and overlapping place cell-like state representations, and experimentally matched parameters. We show how this model maps onto known aspects of hippocampal circuitry and explains substantial variance in the true successor matrix, consequently giving rise to place cells that demonstrate experimentally observed successor representation-related phenomena including backwards expansion on a 1D track and elongation near walls in 2D. Finally, our model provides insight into the observed topographical ordering of place field sizes along the dorsal-ventral axis by showing this is necessary to prevent the detrimental mixing of larger place fields, which encode longer timescale successor representations, with more fine-grained predictions of spatial location.

Predictive maps in rats and humans for spatial navigation

“… 59 Recently, it has been proposed that the hippocampus may implement a system similar to a SR to create a predictive map to guide navigation. 67 , 68 , 71 , 72 Here, we find behavioral evidence to support the proposal that both rats and humans use such a predictive map to guide flexible navigation behavior. This match of the rodent behavior to SR agents is consistent with evidence that rats can carefully evaluate different options for navigation.…”

Predictive Maps in Rats and Humans for Spatial Navigation

“…SR stores a matrix of the possible transitions within the environment and integrates this with information about reward 58 . Recently, it has been proposed that the hippocampus may implement a system similar to a successor representation to create a predictive map to guide navigation 66, 67, 70, 71 . Here we find behavioural evidence to support the proposal that both rats and humans use such a predictive map to guide flexible navigation behaviour.…”

RatInABox: A toolkit for modelling locomotion and neuronal activity in continuous environments

“…Not only is this data cheaper, faster and easier to obtain than experimental data (no rats required), but it can be flexibly hand-designed to rapidly prove or disprove theoretical propositions. Many past[6, 7] and recent[8, 9, 10, 11, 12, 13, 14] models have relied on artificially generated movement trajectories and neural data.…”