Relative luminance and binocular disparity preferences are correlated in macaque primary visual cortex, matching natural scene statistics

Abstract

Humans excel at inferring information about 3D scenes from their 2D images projected on the retinas, using a wide range of depth cues. One example of such inference is the tendency for observers to perceive lighter image regions as closer. This psychophysical behavior could have an ecological basis because nearer regions tend to be lighter in natural 3D scenes. Here, we show that an analogous association exists between the relative luminance and binocular disparity preferences of neurons in macaque primary visual cortex. The joint coding of relative luminance and binocular disparity at the neuronal population level may be an integral part of the neural mechanisms for perceptual inference of depth from images.vision | neuronal coding | stereopsis | illusion T he properties of the natural environment are essential to understanding behavior (1). The study of the statistics of natural images has been instrumental in advancing our understanding of the visual system. Examples are numerous: Center-surround receptive fields of retinal ganglion cells can be understood in terms of their whitening effects on highly self-correlated natural image signals (2). In primary visual cortex (V1), the need to further separate visual signals into their underlying causes provides a potential sparse code explanation for the wavelet-like receptive fields of V1 simple cells (3, 4).Thus far, knowledge of the statistics of natural images has been most useful for understanding how images are represented and transmitted in the early visual pathway (5). However, efficient image representation is only one of many goals of the visual system. To effectively study perceptual inference in the visual system, we must have an understanding of the joint statistics of images together with their perceptual goals. In particular, to understand how depth and 3D shape are inferred in the brain, we need to first understand the statistical trends that exist between natural images and their underlying 3D structure.Statistical trends in natural scenes often manifest as a perceptual bias in psychophysical studies (6-12). One perceptual bias that has been observed is that, all other things being equal, humans perceive lighter image regions as being closer. This perceptual effect was first discovered by Leonardo da Vinci who said, "Among bodies equal in size and distance, that which shines the more brightly seems to the eye nearer," (ref. 13, p. 332). Over the last century, a number of psychophysical studies have characterized this relationship between relative luminance and depth (14-21). We have previously shown that a corresponding negative correlation (r = −0.14) between image intensity and depth (r = −0.24 for log intensity vs. log depth) was found in the statistics of natural scenes (22). This result suggests that the perceptual bias in the psychophysical tests has an ecological basis and is revealing a statistical trend stored within our visual system. Such a trend could be used by the visual system to infer depth in a scene when binocular and other visual ...

Perceived Depth in Natural Images Reflects Encoding of Low-Level Luminance Statistics

“…Previous neurophysiological work has identified cells in primary visual cortex (V1) that are jointly tuned for both relative luminance (increments and decrements) and depth (via binocular disparities). This previous work reported systematic biases in the joint tuning properties of these cells but did not directly compare these biases to the statistical distributions of luminance and depth in natural scenes (Samonds et al, 2012). We sought to compare these neurophysiology data with the information avail- able in natural scenes to determine whether these cell tunings indeed reflect an efficient encoding of the available depth information (Ganguli and Simoncelli, 2010).…”

“…In motion processing, recent work has suggested that ON/OFF segregation is maintained and used to exploit higher-order patterns that are created by motion in natural scenes (Clark et al, 2014). In depth processing, as discussed in the Introduction, neurons involved in stereo-depth computations are jointly tuned for luminance with an overall correlation consistent with natural scenes (Samonds et al, 2012).…”

“…We asked whether the correlation between depth and luminance in natural scenes could create differences between the depths encoded in these relative luminance pathways (for bright and dark). We separately measured the distribution of depths produced by the natural environment for light increments and decrements and found differences that are well matched to biases in the joint tuning preferences of cells in primary visual cortex (Samonds et al, 2012), suggesting a potential mechanism for implicitly encoding two separate prior distributions based on whether a point is bright or dark relative to the surroundings. We also conducted a perceptual experiment using natural scenes, rather than synthetic patterns, as stimuli.…”

“…For example, color and luminance changes tend to occur along with depth changes, and convex edges tend to belong to near surfaces (Burge et al, 2010;Su et al, 2013). Previous work showed that natural scenes also contain a negative correlation between luminance and depth: darker regions tend to be farther away than brighter regions (Potetz andLee, 2003, 2006;Samonds et al, 2012). This correlation suggests that luminance could provide the basis for another prior assumption about depth.…”

“…Neurophysiology data were reanalyzed from Samonds et al (2012), who measured the tuning properties of neurons in macaque primary visual cortex for both relative luminance and depth (via binocular disparity); their Figure 3d shows histograms of disparity preference grouped by "luminance index" (a normalized ratio of a cell's mean firing rate for light increments vs decrements …”