Surface color perception and light field estimation in 3D scenes

Abstract

Previous research on surface color perception has typically used Mondrian stimuli consisting of a small number of matte surface patches in a plane perpendicular to the line of sight. In such scenes, reliable estimation of the color of a surface is a difficult if not impossible computational problem (Maloney, 1999). In three-dimensional scenes consisting of surfaces at many different orientations it is at least in theory possible to estimate surface color. However, the difficulty of the problem increases, in part, because the effective illumination incident on each surface (the light field) now depends on surface orientation and location. We review recent work in multiple laboratories that examines the degree to which the human visual system discounts the light field in judging matte surface lightness and color and how the visual system estimates the flow of light in a scene. The light fieldThe spectral power distribution of light emitted by the sun is almost constant. The variations in daylight (Figure 4.1) we experience over the course of a day and with changes in seasons is due to the interaction of sunlight with the Earth's atmosphere Vision in 3D Environments, ed. L. R. Harris and M. Jenkin.

Accuracy and speed of material categorization in real-world images

“…Numerous studies have examined how the visual system estimates reflectance and geometric shape properties of surfaces in the world. These studies have shown that human observers can reliably estimate certain aspects of surface reflectance such as color, albedo, and gloss (for reviews, see Gilchrist, 2006;Adelson, 2008;Brainard, 2009;Shevell & Kingdom, 2010;Anderson, 2011;Maloney, Gerhard, Boyaci, & Doerschner, 2011;Fleming, 2012). Observers can also estimate surface shape up to certain ambiguities (for a review, see Todd, 2004).…”

“…The nature of this processing remains largely unknown (Gilchrist, 2006;Adelson, 2008;Brainard, 2009;Shevell & Kingdom, 2010;Anderson, 2011;Maloney et al, 2011;Fleming, 2012Fleming, , 2014. In our experiments, we examined the role of four surface properties in material categorization: color, gloss, texture, and shape.…”

Perception of light source distance from shading patterns

“…Despite the remarkable constancy that we experience in everyday life, these studies show that color (Katz, 1935;D. H. Brainard & Wandell, 1992;Hurlbert, 1998;Gegenfurtner, 2003;Maloney, Gerhard, Boyaci, & Doereschner, 2010;Foster, 2011), lightness (Gilchrist, 1977;Gilchrist, Delman, & Jacobsen, 1983;Gerhard & Maloney, 2010a), shape perception (Berbaum, Bever, & Chung, 1983;Koenderink, van Doorn, Christou, & Lappin, 1996;Caniard & Fleming, 2007), and the perception of material properties (Fleming, Dror, & Adelson, 2003;Fleming & Bülthoff, 2005;Motoyoshi & Matoba, 2012;Zhang, de Ridder, & Pont, 2015) depend on the illumination in the scene. Of particular note for the current study are the findings that surface relief and gloss interact in complex and often nonintuitive ways with changes in illumination (Ho, Landy, & Maloney, 2006, 2008Te Pas & Pont, 2009;Wijntjes & Pont, 2010;Fleming, 2012;Marlow, Kim, & Anderson, 2012).…”

Color and material perception: Achievements and challenges

“…Over the past decade, research has increasingly begun to focus on two rich generalizations of the classic "flatmatte" paradigm (for reviews, see Adelson, 2008;Maloney, Gerhard, Boyaci, & Doerschner, 2011). The first generalization concerns the perception of lightness and color in scenes that approximate the three-dimensional environment we live in, with objects that need not be matte or flat and geometrically complex illumination.…”