Frequency-based heuristics for material perception

Abstract

People often make rapid visual judgments of the properties of surfaces they are going to walk on or touch. How do they do this when the interactions of illumination geometry with 3-D material structure and object shape result in images that inverse optics algorithms cannot resolve without externally imposed constraints? A possibly effective strategy would be to use heuristics based on information that can be gleaned rapidly from retinal images. By using perceptual scaling of a large sample of images, combined with correspondence and canonical correlation analyses, we discovered that material properties, such as roughness, thickness, and undulations, are characterized by specific scales of luminance variations. Using movies, we demonstrate that observers' percepts of these 3-D qualities vary continuously as a function of the relative energy in corresponding 2-D frequency bands. In addition, we show that judgments of roughness, thickness, and undulations are predictably altered by adaptation to dynamic noise at the corresponding scales. These results establish that the scale of local 3-D structure is critical in perceiving material properties, and that relative contrast at particular spatial frequencies is important for perceiving the critical 3-D structure from shading cues, so that cortical mechanisms for estimating material properties could be constructed by combining the parallel outputs of sets of frequency-selective neurons. These results also provide methods for remote sensing of material properties in machine vision, and rapid synthesis, editing and transfer of material properties for computer graphics and animation.

MatMix 1.0: Using optical mixing to probe visual material perception

“…We find that this indeed allows the user to adjust different aspects of the proximal image. For highlights, many authors have already shown how their specific characteristics influence perception of glossiness (see e.g., Anderson, 2011;Giesel & Zaidi, 2013;Motoyoshi et al, 2007). Perception of velvetiness, glitter, and sparkle concern undeveloped topics.…”

Stain on texture: Perception of a dark spot having a blurred edge on textured backgrounds

“…It has been shown that at least some material properties are characterized by specific spatial-frequency components of the surface (Bex & Makous, 2002;Giesel & Zaidi, 2013). For example, Giesel and Zaidi (2013) showed that the apparent water absorbency of a surface could be related to the apparent thickness of the surface, and that the apparent thickness in turn could be modulated by changing the relative energy of the frequency band of 8-15 c/image. This band is close to the 6-24 c/image that was shown to be a critical band for producing the stain-on-texture phenomenon in the present study.…”

Linear Subspaces of the Appearance Space

“…By user studies, Giesel and Zaidi found a relation between certain affordances or material properties and spatial frequency bands in material-images [11]: 0.57-2.29 c/°Inflated and deflated 2. 29-4.28 c/°Deep and flat 6.57-15.14 c/°Soft and rough 15.14-19.42 c/°Sparkling and dull The connection between affordance and spatialfrequencies gives rise to a semantically founded editing scheme by simply enhancing or weakening particular frequency bands [12,29]. Because the underlying physical effects are too complicated, those effects may not be seen as the result of inverse optics [12] and are therefore examples for complex physical operations with a linear representation in the visual cortex.…”

“…29-4.28 c/°Deep and flat 6.57-15.14 c/°Soft and rough 15.14-19.42 c/°Sparkling and dull The connection between affordance and spatialfrequencies gives rise to a semantically founded editing scheme by simply enhancing or weakening particular frequency bands [12,29]. Because the underlying physical effects are too complicated, those effects may not be seen as the result of inverse optics [12] and are therefore examples for complex physical operations with a linear representation in the visual cortex. It is striking, that those manipulations cover nearly the whole frequency range of the visual cortex (figure 4).…”

“…While there is evidence, that the frequency-bands are subject to a recognition step [12] and consecutively to a scaling step in the visual cortex, according to the original perceptional studies, an edge length of a material patch should cover a viewing angle of 3.5 • , this corresponds to an observerdistance of approximately 82 cm. We will confine to the roughen and the undulation operation.…”