Disparity processing in primary visual cortex

Henriksen, Sid; Tanabe, Seiji; Cumming, Bruce G.

doi:10.1098/rstb.2015.0255

Cited by 25 publications

(21 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the quarter-century since then, this binocular energy model has become the canonical description of the early stages of disparity encoding. It has been extensively tested against real V1 neurons and although modifications are certainly required, the basic principle underlying the energy model has been vindicated (Cumming & DeAngelis, 2001;Henriksen, Tanabe, & Cumming, 2016;Ohzawa, 1998;Read, 2005); notably, the model has made successful predictions about the response of real neurons to impossible stimuli (Cumming & Parker, 1997). Indeed, stereopsis has become one of the areas where we have the clearest understanding of how perceptual experience relates to early cortical encoding (Parker, 2007;Read, 2014;Roe, Parker, Born, & DeAngelis, 2007).…”

Section: Introductionmentioning

confidence: 99%

A stimulus artefact undermines the evidence for independent ON and OFF channels in stereopsis

Read

Cumming

2018

Preprint

View full text Add to dashboard Cite

Early vision proceeds through distinct ON and OFF channels, which encode luminance increments and decrements respectively. It has been argued that these channels also contribute separately to stereoscopic vision. This is based on the fact that observers perform better on a noisy disparity discrimination task when the stimulus is a random-dot pattern consisting of equal numbers of black and white dots (a "mixed-polarity stimulus", argued to activate both ON and OFF stereo channels), than when it consists of all-white or all-black dots ("samepolarity", argued to activate only one). However, it is not clear how this theory can be reconciled with our current understanding of disparity encoding. Recently, a binocular convolutional neural network was able to replicate the mixed-polarity advantage shown by human observers, even though it was based on linear filters and contained no mechanisms which would respond separately to black or white dots. Here, we show that the stimuli used in all these experiments contain a subtle artefact. The interocular correlation between left and right images is actually lower for the same-polarity stimuli than for mixed-polarity stimuli with the same amount of disparity noise applied to the dots. Since our current theories suggest stereopsis is based on a correlation-like computation in primary visual cortex, it is then unsurprising that performance was better for the mixed-polarity stimuli. We conclude that there is currently no evidence supporting separate ON and OFF channels in stereopsis.

show abstract

Section: Introductionmentioning

confidence: 99%

A stimulus artefact undermines the evidence for independent ON and OFF channels in stereopsis

Read

Cumming

2018

Preprint

View full text Add to dashboard Cite

show abstract

“…For decades, theorists have proposed mathematical principles for computing depth from binocular disparity (Marr and Poggio, 1976;Longuet-Higgins, 1981), and experimental studies using animals have revealed how neurons in visual cortex encode depth from binocular disparity (Barlow et al, 1967;Ohzawa et al, 1990;Cumming and DeAngelis, 2001;Parker, 2007;Henriksen et al, 2016). Computational models (Lehky et al, 1990;Tsai and Victor, 2003) have demonstrated that depth can be decoded from biologically plausible neural representations of binocular disparity.…”

Section: Introductionmentioning

confidence: 99%

Gain Modulation as a Mechanism for Coding Depth from Motion Parallax in Macaque Area MT

2017

View full text Add to dashboard Cite

Observer translation produces differential image motion between objects that are located at different distances from the observer's point of fixation [motion parallax (MP)]. However, MP can be ambiguous with respect to depth sign (near vs far), and this ambiguity can be resolved by combining retinal image motion with signals regarding eye movement relative to the scene. We have previously demonstrated that both extra-retinal and visual signals related to smooth eye movements can modulate the responses of neurons in area MT of macaque monkeys, and that these modulations generate neural selectivity for depth sign. However, the neural mechanisms that govern this selectivity have remained unclear. In this study, we analyze responses of MT neurons as a function of both retinal velocity and direction of eye movement, and we show that smooth eye movements modulate MT responses in a systematic, temporally precise, and directionally specific manner to generate depth-sign selectivity. We demonstrate that depth-sign selectivity is primarily generated by multiplicative modulations of the response gain of MT neurons. Through simulations, we further demonstrate that depth can be estimated reasonably well by a linear decoding of a population of MT neurons with response gains that depend on eye velocity. Together, our findings provide the first mechanistic description of how visual cortical neurons signal depth from MP.

show abstract

“…Generalised versions of the BEM (GBEM), which allow for arbitrary filters and nonlinearities, as well as suppressive subunits, can capture a diverse set of tuning curves, including the attenuated response to anticorrelated stimuli [Nieder and Wagner, 2000, Read et al, 2002, Tanabe and Cumming, 2008, Tanabe et al, 2011, Henriksen et al, 2016c, Goncalves and Welchman, 2017]. However, none of these studies demonstrate that real cells obtain their disparity tuning through a GBEM-type mechanism.…”

Section: Introductionmentioning

confidence: 99%

Current models cannot account for V1's specialisation for binocular natural image statistics

Henriksen

Butts

Read

et al. 2018

Preprint

Self Cite

View full text Add to dashboard Cite

1A long-standing observation about primary visual cortex (V1) is that the stimulus selectivity of 2 neurons can be well explained with a cascade of linear computations followed by a nonlinear rec-3 tification stage. This framework remains highly influential in systems neuroscience and has also 4 inspired recent efforts in artificial intelligence. The success of these models include describing 5 the disparity-selectivity of binocular neurons in V1. Some aspects of real neuronal disparity re-6 sponses are hard to explain with simple linear-nonlinear models, notably the attenuated response 7 of real cells to "anticorrelated" stimuli which violate natural binocular image statistics. General 8 linear-nonlinear models can account for this attenuation, but no one has yet tested whether they 9 quantitatively match the response of real neurons. Here, we exhaustively test this framework using 10 recently developed optimisation techniques. We show that many cells are very poorly characterised 11 by even general linear-nonlinear models. Strikingly, the models can account for neuronal responses 12 to unnatural anticorrelated stimuli as well as to most natural, correlated stimuli. However, the 13 models fail to capture the particularly strong response to binocularly correlated stimuli at the pre-14 ferred disparity of the cell. Thus, V1 neurons perform an amplification of responses to correlated 15 stimuli which cannot be accounted for by a linear-nonlinear cascade. The implication is that even 16 simple stimulus selectivity in V1 requires more complex computations than previously envisaged. 17 1 Significance statement 18A long-standing question in sensory systems neuroscience is whether the computations performed 19 by neurons in primary visual cortex can be described by repeated elements of linear-nonlinear units 20 (a linear filtering/pooling stage followed by a subsequent output nonlinearity, such as a squaring). 21 This question goes back to the Nobel-prize winning work by Hubel & Wiesel who argued that 22 orientation selectivity in V1 can qualitatively be explained in this way. In this paper, we show 23 that V1 neurons have an amplification of their response to stimuli which are contrast matched 24 in the two eyes, and that the recovered models cannot describe this property. We argue that 25 this likely represents more sophisticated computations than can be compactly described by the 26 linear-nonlinear cascade framework. 27In their classic model for orientation selectivity in the visual cortex, Hubel and Wiesel (1962) pro-29 posed that simple cells performed a linear operation on the retinal image. A nonlinear relationship 30 between the filter response and the spike rate was sufficient to account for the observed selectivity 31 (a linear-nonlinear, or LN, model of simple cells [Hubel and Wiesel, 1962]). Complex cell properties 32 such as position invariance could be modelled as the sum of several such LN "subunits" followed 33 by a second nonlinearity. This process, where the output of one set of LN models forms...

show abstract

Disparity processing in primary visual cortex

Cited by 25 publications

References 49 publications

A stimulus artefact undermines the evidence for independent ON and OFF channels in stereopsis

A stimulus artefact undermines the evidence for independent ON and OFF channels in stereopsis

Gain Modulation as a Mechanism for Coding Depth from Motion Parallax in Macaque Area MT

Current models cannot account for V1's specialisation for binocular natural image statistics

Contact Info

Product

Resources

About