A Sensory-Motor Theory of the Neocortex based on Active Predictive Coding

Abstract

We propose that the neocortex implements active predictive coding (APC), a form of predictive coding that incorporates hierarchical dynamics and actions. In this model, each neocortical area estimates both sensory states and actions, and the cortex as whole learns to predict the sensory consequences of actions at multiple hierarchical levels. ''Higher'' cortical areas maintain more abstract representations at larger spatiotemporal scales compared to ''lower'' areas. Feedback from higher areas modulate the dynamics of both state and action networks in lower areas. This allows the cortical network to model the complex dynamics and physics of the world in terms of simpler compositional elements (state transition functions). Simultaneously, current higher level goals invoke sequences of lower level sub-goals and actions, allowing the network to solve complex planning problems by composing simpler solutions. Planning ("system 2" thinking) in turns allows the network to learn, over time, perception-to-action mappings (policies; "system 1" thinking) at multiple abstraction levels. We provide examples from simulations illustrating how the same APC architecture can solve problems that, at first blush, seem very different from each other: (1) how do we recognize an object and its parts using eye movements? (2) why does perception seem stable despite eye movements? (3) how do we learn compositional representations, e.g., part-whole hierarchies, and nested reference frames for equivariant vision? (4) how do we model the "physics" of a complex environment by decomposing it into simpler components? (5) how do we plan actions in a complex domain to achieve a goal by composing sequences of sub-goals and simpler actions? and (6) how do we form episodic memories of sensory-motor experiences? We propose a mapping of the APC network to the laminar architecture of the cortex and suggest possible roles for cortico-cortical, cortico-thalamic, cortico-hippocampal and cortico-subcortical pathways.

A non-Hebbian code for episodic memory

“…In mammals, world models are thought to include complex state spaces that are acquired progressively (e.g. via gradual Hebbian plasticity) in neocortex (33, 37–41). Four key properties of such state spaces are relevant to our memory model: (1) states and their transition statistics can be represented as a directed graph—a standard approach in Reinforcement Learning (RL), for example; (2) states evolve in response to a variety of signals but can also be maintained without continued sensory input (e.g.…”

“…5). In contrast, most existing models treat one-shot episodic memory as a generic buffer for temporarily storing arbitrary information (6, 7, 34, 36, 39, 41, 49, 67, 68) (typically through Hebbian plasticity), with policies existing in a separate function space. In episodic RL, for instance, episodes are stored as memorized action sequences that can be sampled to efficiently approximate state-action values to shape policy (34, 36).…”