Current models cannot account for V1's specialisation for binocular natural image statistics

Abstract

1A long-standing observation about primary visual cortex (V1) is that the stimulus selectivity of 2 neurons can be well explained with a cascade of linear computations followed by a nonlinear rec-3 tification stage. This framework remains highly influential in systems neuroscience and has also 4 inspired recent efforts in artificial intelligence. The success of these models include describing 5 the disparity-selectivity of binocular neurons in V1. Some aspects of real neuronal disparity re-6 sponses are hard to explain with simple linear-nonlinear models, notably the attenuated response 7 of real cells to "anticorrelated" stimuli which violate natural binocular image statistics. General 8 linear-nonlinear models can account for this attenuation, but no one has yet tested whether they 9 quantitatively match the response of real neurons. Here, we exhaustively test this framework using 10 recently developed optimisation techniques. We show that many cells are very poorly characterised 11 by even general linear-nonlinear models. Strikingly, the models can account for neuronal responses 12 to unnatural anticorrelated stimuli as well as to most natural, correlated stimuli. However, the 13 models fail to capture the particularly strong response to binocularly correlated stimuli at the pre-14 ferred disparity of the cell. Thus, V1 neurons perform an amplification of responses to correlated 15 stimuli which cannot be accounted for by a linear-nonlinear cascade. The implication is that even 16 simple stimulus selectivity in V1 requires more complex computations than previously envisaged. 17 1 Significance statement 18A long-standing question in sensory systems neuroscience is whether the computations performed 19 by neurons in primary visual cortex can be described by repeated elements of linear-nonlinear units 20 (a linear filtering/pooling stage followed by a subsequent output nonlinearity, such as a squaring). 21 This question goes back to the Nobel-prize winning work by Hubel & Wiesel who argued that 22 orientation selectivity in V1 can qualitatively be explained in this way. In this paper, we show 23 that V1 neurons have an amplification of their response to stimuli which are contrast matched 24 in the two eyes, and that the recovered models cannot describe this property. We argue that 25 this likely represents more sophisticated computations than can be compactly described by the 26 linear-nonlinear cascade framework. 27In their classic model for orientation selectivity in the visual cortex, Hubel and Wiesel (1962) pro-29 posed that simple cells performed a linear operation on the retinal image. A nonlinear relationship 30 between the filter response and the spike rate was sufficient to account for the observed selectivity 31 (a linear-nonlinear, or LN, model of simple cells [Hubel and Wiesel, 1962]). Complex cell properties 32 such as position invariance could be modelled as the sum of several such LN "subunits" followed 33 by a second nonlinearity. This process, where the output of one set of LN models forms...

Stereopsis without correspondence

“…Intuitively, this is because as more simple cells are added, it becomes progressively more challenging to swap in a different monocular image which produces the same activation in all units. The large response of V1 neurons to their preferred disparity [57][58][59] and their lack of response towards 'anti-correlated' stimuli with artificially low matches [60] has recently been modelled by a convolution model combining very many simple cells [61]. Repeated iterations of this process may also explain the properties of neurons in higher areas such as IT, where the response to anti-correlated stimuli is nearly abolished [62].…”