Sensors for impossible stimuli may solve the stereo correspondence problem

Abstract

A fundamental challenge of binocular vision is that objects project to different positions on the two retinas (binocular disparity). Neurons in visual cortex show two distinct types of tuning to disparity: position and phase disparity, due to differences in receptive field location and profile respectively. Here, we point out that phase disparity does not occur in natural images. Why, then, should the brain encode it? We propose that phase disparity detectors help work out which feature in the left eye corresponds to a given feature in the right. This correspondence problem is bedeviled by false matches: regions of the image which look similar but do not correspond to the same object. We show that phase-disparity neurons tend to be more strongly activated by false matches. Thus, they may act as "lie detectors", enabling the true correspondence to be deduced by a process of elimination.Over the past 35 years, neurophysiologists have mapped the response properties of binocular neurons in primary visual cortex and elsewhere in considerable detail. A mathematical model, the stereo energy model 1 , has been developed, which successfully describes many of their properties. Within this model, binocular neurons can encode disparity in two basic ways: phase disparity (Fig. 1a), in which receptive fields differ in the arrangement of their ON and OFF regions, but not in retinal position, and position disparity (Fig. 1b), in which left and right-eye receptive fields differ in their position on the retina but not in their profile 2 . Several recent studies in V1 3-6 have concluded that most disparity-selective cells are hybrid 7 , showing both position and phase disparity. These neurophysiological data present a challenge to computational models of stereopsis. Why does the brain devote computational resources to encoding disparity twice over, once through position and once through phase?Phase disparity presents a particular puzzle, since it does not correspond to anything experienced in natural viewing. For a surface such as a wall in front of the observer, where disparity is locally uniform, the two eyes' images of a given patch on the surface are related by a simple position shift on the retina (Fig. 1b). For an inclined surface, with a linear disparity gradient, the two image patches are also compressed and/or rotated with respect to one another: i.e., they differ in spatial frequency and/or orientation (Fig. 1c). Higher-order changes in disparity, such as produced by curved surfaces, produce images whose spatial frequency and orientation differences vary across the retina (Fig. 1d 8 ). Disparity discontinuities, which occur at object boundaries, produce different disparities in different regions of the retina 9 . However, phase disparity neurons do not appear to be constructed to detect any of these possible situations. They respond optimally to stimuli in which the left and right eye's image are related by a constant shift in Fourier phase, i.e. each Fourier component is displaced by an amount proportional to it...

Evidence and Counterevidence in Motion Perception

“…Our data support the view that such a counterevidence strategy is implemented by direction-selective V1 neurons. A counterevidence strategy was also proposed by Read and Cumming (2007) to solve the correspondence problem in stereopsis. They argued that phase-disparity detectors in V1 provide counterevidence, or in their terms act as "lie-detectors", to unmask false positives signaled by position-disparity detectors.…”

Phase-differencing in Stereo Vision - Solving the Localisation Problem

“…It captures all structural information of the pyramid, but no surfaces. The horizontal lines of the pyramid can also be detected, perhaps with a residual disparity component, but only if the exact geometry of the projections in our eyes is taken into account (Read and Cumming, 2007). The spikes and curved parts at the line ends in Fig.…”

“…For an overview of alternative biological disparity models see (Read and Cumming, 2007). In the next two sections we present a brief overview of earlier work, including Sanger's, to illustrate the "sting," the constraints which have been and still are applied in estimating disparity, and some important assumptions which should be taken into account.…”

Disparity processing in primary visual cortex

“…These stimuli are 'unnatural' in the sense that it is impossible to achieve anticorrelation in natural viewing. 3 The viewing geometry imposes some close relationships between spatial frequency and interocular phase difference [11,13,21], and anticorrelation represents the largest possible deviation from these naturally occurring relationships [21]. It is presumably the 'unnatural' binocular structure that produces the weak responses in real neurons, but the BEM is not sensitive to this.…”