Imagery May Arise from Associations Formed through Sensory Experience: A Network of Spiking Neurons Controlling a Robot Learns Visual Sequences in Order to Perform a Mental Rotation Task

Abstract

Mental imagery occurs “when a representation of the type created during the initial phases of perception is present but the stimulus is not actually being perceived.” How does the capability to perform mental imagery arise? Extending the idea that imagery arises from learned associations, we propose that mental rotation, a specific form of imagery, could arise through the mechanism of sequence learning–that is, by learning to regenerate the sequence of mental images perceived while passively observing a rotating object. To demonstrate the feasibility of this proposal, we constructed a simulated nervous system and embedded it within a behaving humanoid robot. By observing a rotating object, the system learns the sequence of neural activity patterns generated by the visual system in response to the object. After learning, it can internally regenerate a similar sequence of neural activations upon briefly viewing the static object. This system learns to perform a mental rotation task in which the subject must determine whether two objects are identical despite differences in orientation. As with human subjects, the time taken to respond is proportional to the angular difference between the two stimuli. Moreover, as reported in humans, the system fills in intermediate angles during the task, and this putative mental rotation activates the same pathways that are activated when the system views physical rotation. This work supports the proposal that mental rotation arises through sequence learning and the idea that mental imagery aids perception through learned associations, and suggests testable predictions for biological experiments.

Mechanisms of Winner-Take-All and Group Selection in Neuronal Spiking Networks

“…Such learning process will modify the synapses between these two excitatory types so that a selected E and I neurons (the WTA group) will respond to preferred input patterns more quickly for practical applications. We have demonstrated these in previous reports (Chen et al, 2013 ; McKinstry et al, 2016 ) where the same WTA network structures were implemented in a humanoid robot to process real world complex visual inputs, to learn visual-motor association and sequencing, and to accomplish a “mental rotation" and delayed-match-to-sample task.…”

“…This WTA network has been implemented into a robot that accomplished a sequence learning and mental rotation task (McKinstry et al, 2016). In our spiking models each neuron type has very detailed biological parameters to model different neuronal transmitters and receptor types similar to previous work (Izhikevich and Edelman, 2008).…”

“…In a recent report we presented a robust and more biologically-realistic WTA network structure with distinct excitatory and inhibitory populations with arbitrary number of units (Chen et al, 2013 ). This WTA network has been implemented into a robot that accomplished a sequence learning and mental rotation task (McKinstry et al, 2016 ). In our spiking models each neuron type has very detailed biological parameters to model different neuronal transmitters and receptor types similar to previous work (Izhikevich and Edelman, 2008 ).…”

A Biologically Plausible Neuron Model of Mental Rotation

“…At this point, I feel it is important to state that I am not the first to computationally model mental rotation in a biologically plausible way. Of particular note is the work of McKinstry et al, (2016), who used a network of spiking neurons to perform the Shepard and Metzler (1971) paradigm with a humanoid robot. However, despite the biological accuracy of the underlying neurons and the use of three-dimensional stimuli, the act of mental rotation was achieved by observing the object rotate, then learning the pattern of neural activation required to perform mental rotations (McKinstry et al, 2016).…”

“…Of particular note is the work of McKinstry et al, (2016), who used a network of spiking neurons to perform the Shepard and Metzler (1971) paradigm with a humanoid robot. However, despite the biological accuracy of the underlying neurons and the use of three-dimensional stimuli, the act of mental rotation was achieved by observing the object rotate, then learning the pattern of neural activation required to perform mental rotations (McKinstry et al, 2016). Although this parallels a known strategy (see Kosslyn et al, 2001), it is not the dominant one.…”