A Single Mechanism Can Account for Human Perception of Depth in Mixed Correlation Random Dot Stereograms

Abstract

In order to extract retinal disparity from a visual scene, the brain must match corresponding points in the left and right retinae. This computationally demanding task is known as the stereo correspondence problem. The initial stage of the solution to the correspondence problem is generally thought to consist of a correlation-based computation. However, recent work by Doi et al suggests that human observers can see depth in a class of stimuli where the mean binocular correlation is 0 (half-matched random dot stereograms). Half-matched random dot stereograms are made up of an equal number of correlated and anticorrelated dots, and the binocular energy model—a well-known model of V1 binocular complex cells—fails to signal disparity here. This has led to the proposition that a second, match-based computation must be extracting disparity in these stimuli. Here we show that a straightforward modification to the binocular energy model—adding a point output nonlinearity—is by itself sufficient to produce cells that are disparity-tuned to half-matched random dot stereograms. We then show that a simple decision model using this single mechanism can reproduce psychometric functions generated by human observers, including reduced performance to large disparities and rapidly updating dot patterns. The model makes predictions about how performance should change with dot size in half-matched stereograms and temporal alternation in correlation, which we test in human observers. We conclude that a single correlation-based computation, based directly on already-known properties of V1 neurons, can account for the literature on mixed correlation random dot stereograms.

Emergence of binocular disparity selectivity through Hebbian learning

“…Interestingly, when the complexity of the decoder is increased slightly by including quadratic terms (blue), one observes a substantial increase in discrimination performance. This is not surprising as our neurones are linear units similar to simple cells, and a non-linear processing of their activity makes the decoding units conceptually closer to sharply tuned complex cells [52][53][54] . These results show that disparity encoding in the early visual system does not need supervised training, and an unsupervised feedforward Hebbian process can lead to neural units whose responses can be interpreted in terms of 3D percept through downstream processing.…”

Specialization of mid-tier stages of dorsal and ventral pathways in stereoscopic processing

“…The disparity energy model is a classic implementation of the correlation-based detector (Cumming and Parker, 1997;Ohzawa et al, 1990). Additional nonlinearity can transform the correlation-based detector to the match-based detector under some conditions (Doi and Fujita, 2014;Henriksen et al, 2016a). (A) Three RDSs with graded anticorrelation.…”

“…Adding expansive nonlinearity to disparity energy models can achieve the transformation under some conditions (Doi and Fujita, 2014;Lippert and Wagner, 2001). Although the additional nonlinearity must be only part of the full mechanism, it has psychophysical (Doi et al, 2013;Henriksen et al, 2016a) and physiological support (Henriksen et al, 2016b). A key constraint of this mechanism is that the energy-model units should have even-symmetric, but not odd-symmetric, disparity tuning.…”

Near-optimal combination of disparity across a log-polar scaled visual field

“…The spatial frequency of depth variation at which peak 64 sensitivity occurs also decreases from the fovea to the peripheral visual field (Figure 1e, 65 F 2,18 = 15.87, p = 1.1 × 10 −4 ), whereas the bandwidth of disparity tuning remains 66 constant (Figure 1f, F 2,18 = 0.2, p = 0.82). 67 Humans integrate disparity information across the visual field 68 in a near-optimal fashion 69 Figure 1a (bottom plot) shows how disparity sensitivity for the full field stimuli (black) 70 is the envelope of the disparity sensitivities estimated in the restricted visual field 71 conditions. Additionally, Figure 1b (top plot) shows how disparity sensitivity for stimuli 72 spanning the whole visual field (black) approaches the level of sensitivity predicted from 73 the MLE optimal combination of disparity sensitivity across the separate portions of the 74 visual field (magenta, following [22], see methods section for precise mathematical 75 formulation).…”

“…by considering that d = ∆ψ/f s . By taking into account the extensions of the binocular 444 energy model proposed in [67,68], we apply a static non-linearity to the complex cell 445 response described in Eq. 12.…”