A Model for Encoding Multiple Object Motions and Self-Motion in Area MST of Primate Visual Cortex

Abstract

Many cells in the dorsal part of the medial superior temporal (MST) region of visual cortex respond selectively to specific combinations of expansion/contraction, translation, and rotation motions. Previous investigators have suggested that these cells may respond selectively to the flow fields generated by self-motion of an observer. These patterns can also be generated by the relative motion between an observer and a particular object. We explored a neurally constrained model based on the hypothesis that neurons in MST partially segment the motion fields generated by several independently moving objects. Inputs to the model were generated from sequences of ray-traced images that simulated realistic motion situations, combining observer motion, eye movements, and independent object motions. The input representation was based on the response properties of neurons in the middle temporal area (MT), which provides the primary input to area MST. After applying an unsupervised optimization technique, the units became tuned to patterns signaling coherent motion, matching many of the known properties of MST cells. The results of this model are consistent with recent studies indicating that MST cells primarily encode information concerning the relative three-dimensional motion between objects and the observer.

A neural model of visually guided steering, obstacle avoidance, and route selection.

“…The Perrone and Stone model also requires a large number of templates to handle optic flow fields generated by scenes with different depth structures, and it does not explain how the templates could be learned. Zemel and Sejnowski (1998) showed how a model could be trained using gradient descent to mimic MSTlike responses in a hidden layer and encode both object motion and heading. Gradient descent training methods for neural networks require non-local transport of learned weights, for which there is no known biological evidence.…”

“…MST & heading perception during eye rotation Zemel & Sejnowski (1998) Unsupervised learning MST heading & object motion Grossberg, Mingolla, & Pack (1999) Neural net using log-polar optic flow MST heading & optic flow classification; translational heading perception Wagner (2004) Computer vision, log-polar optic flow . Screenshot from our 3D simulation environment.…”

Cortical dynamics of navigation and steering in natural scenes: Motion-based object segmentation, heading, and obstacle avoidance

“…Thus, although Bayerl and Neumann (2004) present interesting results, the model would need to be modified to account for the human behavioral data and primate neurophysiological that ViSTARS explains. Zemel and Sejnowski (1998) presented a model in which MSTd codes not only observer motion with respect to the environment (heading) but also the relative motion between an observer and a particular object. Inputs were ray-tracings of image sequences depicting observer motion, eye movements, and object motion.…”

Attending to Motion: Localizing and Classifying Motion Patterns in Image Sequences

“…Here, the motion domain is chosen in order to demonstrate that STM can indeed operate as desired with realistic representations because enough is known about motion processing to enable a reasonable attempt at defining the feedforward pyramid. In addition, the effort is unique because it seems that no past model presented a motion hierarchy plus attention to motion [34,35,36,37,38,39,40,41,42]. The remainder of this paper will focus on this issue.…”