The emulation theory of representation: Motor control, imagery, and perception

Abstract

The emulation theory of representation is developed and explored as a framework that can revealingly synthesize a wide variety of representational functions of the brain. The framework is based on constructs from control theory (forward models) and signal processing (Kalman filters). The idea is that in addition to simply engaging with the body and environment, the brain constructs neural circuits that act as models of the body and environment. During overt sensorimotor engagement, these models are driven by efference copies in parallel with the body and environment, in order to provide expectations of the sensory feedback, and to enhance and process sensory information. These models can also be run off-line in order to produce imagery, estimate outcomes of different actions, and evaluate and develop motor plans. The framework is initially developed within the context of motor control, where it has been shown that inner models running in parallel with the body can reduce the effects of feedback delay problems. The same mechanisms can account for motor imagery as the off-line driving of the emulator via efference copies. The framework is extended to account for visual imagery as the off-line driving of an emulator of the motor-visual loop. I also show how such systems can provide for amodal spatial imagery. Perception, including visual perception, results from such models being used to form expectations of, and to interpret, sensory input. I close by briefly outlining other cognitive functions that might also be synthesized within this framework, including reasoning, theory of mind phenomena, and language.

Towards a computational theory of experience

“…In the simplest representational scenario, correspondence holds between successive states of the two processes, as well as between their respective timings. In this case, the state-space trajectory of one system unfolds in lockstep with that of the other system, because the dynamics of the two systems are sufficiently close to one another; for example, formal neurons can be wired up into a network whose dynamics would emulate (Grush, 2004) that of the falling rock mentioned above. More interesting are cases in which the correspondence exists on a more abstract level, for instance between a certain similarity structure over some physical variables -out there‖ in the world (e.g., between objects that fall like a rock and those that drift down like a leaf) and a conceptual structure over certain instances of neural activity, as well as cases in which the system emulates aspects of its own dynamics.…”

“…More interesting are cases in which the correspondence exists on a more abstract level, for instance between a certain similarity structure over some physical variables -out there‖ in the world (e.g., between objects that fall like a rock and those that drift down like a leaf) and a conceptual structure over certain instances of neural activity, as well as cases in which the system emulates aspects of its own dynamics. Further still, note that once representational mechanisms have been set in place, they can also be used -offline‖ (Grush, 2004). In all cases, the combinatorics of the world ensures that the correspondence relationship behind instances of representation is highly non-trivial, that is, unlikely to persist purely as a result of a chance configurational alignment between two randomly picked systems (Chalmers, 1994).…”

Motor execution affects action prediction

“…This notion corresponds to the view that sensorimotor simulations enable the extrapolation of future actions (Grush, 2004;Thornton & Knoblich, 2006). Efficient body control requires the estimation of one's own body state prior to movement execution, which is based on internal forward models.…”

Intentional action: from anticipation to goal-directed behavior

“…In this context, the abilities of action execution, its planning and understanding of others' intentions are all described as essentially goal-directed and served by the same representations, which are action-oriented and involve deeply the motor apparatus. Simulative theories of cognition add to this picture the idea that representing means engaging in simulated interaction with the environment and with others, something that could be done by using (online or oZine) the same set of internal models implied in motor control (Grush, 2004). Along similar lines, a few uniWed frameworks have been proposed, based on the idea that an essential part of one's understanding of the dynamics of the environment, and of others' intentions, is in terms of one's own motor repertoire and intentions, with crucial involvement of anticipatory and simulative mechanisms (Gallese, Keysers, & Rizzolatti, 2004;Jeannerod, 2001;Wolpert, Doya, & Kawato, 2003).…”