The lawful imprecision of human surface tilt estimation in natural scenes

Abstract

Estimating local surface orientation (slant and tilt) is fundamental to recovering the threedimensional structure of the environment, but it is unknown how well humans perform this task in natural scenes. Here, with a high-fidelity database of natural stereo-images with groundtruth surface orientation at each pixel, we find dramatic differences in human tilt estimation with natural and artificial stimuli. With artificial stimuli, estimates are precise and unbiased. With natural stimuli, estimates are imprecise and strongly biased. An image-computable normative model grounded in natural scene statistics predicts human bias, precision, and trial-by-trial errors without fitting parameters to the human data. These similarities suggest that the complex human performance patterns with natural stimuli are lawful, and that human visual systems have internalized local image and scene statistics to optimally infer the three-dimensional structure of the environment. The current results help generalize our understanding of human vision from the lab to the real world.

Picture perception reveals mental geometry of 3D scene inferences

“…Measurements such as pose estimation, extended to other object shapes and locations, may help address such long-standing issues. The fronto-parallel bias we find is curious, but has been reported previously for slant of planar surfaces (27) and tilt of oblique objects in natural scenes (28). Could it be based on adaptation to orientation statistics of natural scenes?…”

Learning to see stuff

“…Yet, due to their physical complexity, they pose profound challenges for traditional ‘inverse optics’ theories of perception [6, 7, 8,9 • ,77]. Most theories assume the brain’s goal is to estimate physical quantities, like surface reflectance, orientation or depths [10, 11, 12]. Yet when we perceive complex materials, what exactly is the brain ‘estimating’?…”

Predicting the Partition of Behavioral Variability in Speed Perception with Naturalistic Stimuli

“…The vision and systems neuroscience communities have traditionally focused on understanding how simple forms of stimulus variability (e.g., Poisson or Gaussian white noise) impact performance (Hecht et al, 1942;Burgess et al, 1981;Pelli, 1985;Banks et al, 1987;Frechette et al, 2005). The impact of natural stimulus variability, the variation in light patterns associated with different natural scenes sharing the same latent variable values, has only recently begun to receive significant attention (Geisler and Perry, 2009;Burge and Geisler, 2011Kane et al, 2011;Sebastian et al, 2015Sebastian et al, , 2017Schütt and Wichmann, 2017;Kim and Burge, 2018;Sinha et al, 2018).…”