Temporal Dynamics of Binocular Disparity Processing in the Central Visual Pathway

Abstract

. To solve the stereo correspondence problem (i.e., find the matching features of a visual scene in both eyes), it is advantageous to combine information across spatial scales. The details of how this is accomplished are not clear. Psychophysical studies and mathematical models have suggested various types of interactions across spatial scale, including coarse to fine, fine to coarse, averaging, and population coding. In this study, we investigate dynamic changes in disparity tuning of simple and complex cells in the cat's striate cortex over a short time span. We find that disparity frequency increases and disparity ranges decrease while optimal disparity remains constant, and this conforms to a coarse-to-fine mechanism. We explore the origin of this mechanism by examining the frequency and size dynamics exhibited by binocular simple cells and neurons in the lateral geniculate nucleus (LGN). The results suggest a strong role for a feed-forward mechanism, which could originate in the retina. However, we find that the dynamic changes seen in the disparity range of simple cells cannot be predicted from their left and right eye monocular receptive field (RF) size changes. This discrepancy suggests the possibility of a dynamic nonlinearity or disparity specific feedback that alters tuning or a combination of both mechanisms. I N T R O D U C T I O NStereoscopic depth perception has been studied from theoretical, behavioral, and neurophysiological perspectives. Most behavior work has been aimed at establishing empirical parameters of stereoscopic function and performance levels. Early neurophysiological studies neglected mechanisms and assumed that cells with responses that changed with retinal disparity of the stimulus were depth detectors. More recent physiological work has included theoretical proposals that were tested experimentally. The result is a modified energy model that accounts for basic features of the neural mechanism of stereopsis (e.g., Freeman and Ohzawa 1990;Ohzawa et al. 1990;).An early theoretical proposal was concerned with the correspondence problem in stereopsis (Marr and Poggio 1979). This refers to the ambiguity of correspondence between left and right images that occurs from binocular viewing. The brain must choose the correct depth plane from several possible ones to process appropriate stereoscopic information. To solve this problem, coarse scale disparity matches were proposed to occur first. This would be followed by fine scale matches (Marr and Poggio 1979). In other words, a coarse-to-fine scaling process could be used to provide correct stereoscopic matching. Other theoretical ideas envisioned similar coarse scale disparity matches that were followed by fine scale adjustments (Anderson and Van Essen 1987;Nishihara and Kimura 1987;Quam 1987).These theoretical notions have been explored in behavioral studies. Sensitivity has been found to improve for line length, orientation, curvature and stereoscopic depth over a period of Ն1 s (Watt 1987). The interpretation of these findings may be ...

Temporal Dynamics of Shape Analysis in Macaque Visual Area V2

“…In contrast, Zohary et al (1990) reported increased orientation discriminability as the integration window increased from 60-100 ms, and Ringach et al (1997) reported significant time dependency of orientation tuning using a finer-grained temporal analysis (10 vs. 50 ms bins). In the domain of stereopsis, Menz andFreeman (2003, 2004) showed that binocular disparity tuning in cat visual cortex sharpens with time, consistent with coarse-to-fine processing.…”

“…In the visual system, LGN (lateral geniculate nucleus) neurons temporally decorrelate in response to natural movies, but not to white noise, by a process involving linear temporal filtering characteristics of neurons (Dan et al 1996). In the visual cortex, many previous studies have explored the temporal dynamics of individual cell responses, especially in area V1 (see, e.g., Albrecht et al 2002;Bredfeldt and Ringach 2002;Celebrini et al 1993;DeAngelis et al 1993;Jones and Palmer 1987;Mazer et al 2002;Menz andFreeman 2003, 2004;Müller et al 2001;Ringach et al 1997;Shapley et al 2003) and in the inferior temporal cortex (Edwards et al 2003;Keysers et al 2001;Oram and Perrett 1992;Tovée et al 1993). Recently, Müller et al (2001) analyzed the temporal dynamics of the signal-tonoise ratio in area V1 of the anesthetized monkey and found that V1 cells convey much more information during the initial transients than during later periods of the same duration.…”

“…Second, the firing rate of a visual cortical neuron rarely stays constant during a sustained presentation of a given stimulus; the neuronal response to visual stimulation typically includes an initial transient peak followed by a declining response (Keysers et al 2001;Lisberger and Movshon 1999;Müller et al 1999Müller et al , 2001Oram and Perrett 1992;Tolhurst et al 1980Tolhurst et al , 1983; also see Rieke et al 1998). Explorations of the effects of this latter type of firing rate variation in the visual cortex have largely focused on the temporal dynamics of selectivity for various low-level stimulus characteristics, such as orientation, spatial frequency, contrast, and disparity, notably in area V1 (see, e.g., Albrecht et al 2002;Bredfeldt and Ringach 2002;Frazor et al 2004;Menz andFreeman 2003, 2004;Müller et al 2001;Shapley et al 2003; also see DISCUSSION). Additional studies have examined the temporal dynamics of selectivity for orientation contrast in V1 (Knierim and Van Essen 1992), illusory contours in V2 (Lee and Nguyen 2001), "border ownership " in V2 and V4 (Zhou et al 2000), complex chromatic and achromatic shapes in inferotemporal cortex (Edwards et al 2003;Keysers et al 2001;Oram and Perrett 1992;Tovée et al 1993), and motionbased features in middle temporal (MT), middle superior temporal (MST), and ventral intraparietal (VIP) areas (Cook and Maunsell 2002;Duffy and Wurtz 1997;Pack and Born 2001; also see DISCUSSION).…”

Spatial Reference Frames of Visual, Vestibular, and Multimodal Heading Signals in the Dorsal Subdivision of the Medial Superior Temporal Area

“…However, meaningful changes in tuning properties can occur on much shorter time scales (Ringach et al, 1997;Cottaris and De Valois, 1998;Menz and Freeman, 2004), prompting us to examine the evolution of spatial reference frames over time. For each neuron, we computed DI within a 100 ms window that was stepped through the 2 s stimulus period in increments of 50 ms. We then plotted the mean DI (ϮSEM) across neurons for each time point, separated by stimulus condition (Fig.…”

Disparity processing in primary visual cortex

“…It is also important to note that combining excitation at one scale with inhibition over a broader scale has the effect of 'sharpening' the disparity tuned response of the sum. Consequently, the responses of single neurons do move from coding coarser scales to finer scales over a period of 20-50 ms [7,[26][27][28]. The early coarse-to-fine algorithms imagined this combination happening at a stage after the initial binocular filtering [15].…”