Natural scene statistics predict how humans pool information across space in surface tilt estimation

Abstract

Visual systems estimate the three-dimensional (3D) structure of scenes from information in two-dimensional (2D) retinal images. Visual systems use multiple sources of information to improve the accuracy of these estimates, including statistical knowledge of the probable spatial arrangements of natural scenes. Here, we examine how 3D surface tilts are spatially related in real-world scenes, and show that humans pool information across space when estimating surface tilt in accordance with these spatial relationships. We develop a hierarchical model of surface tilt estimation that is grounded in the statistics of tilt in natural scenes and images. The model computes a global tilt estimate by pooling local tilt estimates within an adaptive spatial neighborhood. The spatial neighborhood in which local estimates are pooled changes according to the value of the local estimate at a target location. The hierarchical model provides more accurate estimates of groundtruth tilt in natural scenes and provides a better account of human performance than the local estimates. Taken together, the results imply that the human visual system pools information about surface tilt across space in accordance with natural scene statistics.

Perceptual consequences of interocular differences in the duration of temporal integration

“…horizontal size ratios of 0.5 or 2.0, depending on whether the surface is slanted left- or right-side back). For more distant and less slanted surfaces, which are more common in natural viewing ( Adams, Elder, Graf, Leyland, Lugtigheid, & Muryy, 2016 ; Backus, Banks, van Ee, & Crowell, 1999 ; Burge, McCann, & Geisler, 2016 ; Kim & Burge, 2018 ; Kim & Burge, 2020 ; Yang & Purves, 2003 ), the ratio tends to be substantially smaller. However, typical natural images have broadband 1/f spectra, and frequencies above the contrast detection threshold typically vary by a factor of 10 or more.…”

An image reconstruction framework for characterizing initial visual encoding

“…Such analyses have often been applied to analyze performance for simple artificial stimuli, assuming that the stimuli to be discriminated are known exactly ( Banks et al, 1987 ; Davila and Geisler, 1991 ) or known statistically with some uncertainty ( Pelli, 1985 ; Geisler, 2018 ). The ideal observer approach has been extended to consider decision processes that learn aspects of the stimuli being discriminated, rather than being provided with these a priori, and extended to handle discrimination and estimation tasks with naturalistic stimuli ( Burge and Geisler, 2011 ; Burge and Geisler, 2014 ; Singh et al, 2018 ; Chin and Burge, 2020 ; Kim and Burge, 2020 ). For a recent review, see Burge, 2020 ; also see Tjan and Legge, 1998 and Cottaris et al, 2019 ; Cottaris et al, 2020 .…”

Stereo slant discrimination of planar 3D surfaces: Frontoparallel versus planar matching

“…In most of these studies, the stimuli were presented in two temporal intervals. In natural viewing, it is probably more typical for humans to be comparing the 3D orientations of surfaces that are densely textured, and that are located within the same scene at different distances ( Burge, McCann, & Geisler, 2016 ; Kim & Burge, 2018 ; Kim & Burge, 2020 ). Here, we measured slant discrimination performance for surfaces that were textured with naturalistic noise (see Figure 2 in Methods).…”