Detection and localization of specular surfaces using image motion cues

Abstract

Successful identification of specularities in an image can be crucial for an artificial vision system when extracting the semantic content of an image or while interacting with the environment. We developed an algorithm that relies on scale and rotation invariant feature extraction techniques and uses motion cues to detect and localize specular surfaces. Appearance change in feature vectors is used to quantify the appearance distortion on specular surfaces, which has previously been shown to be a powerful indicator for specularity (Doerschner et al. in Curr Biol, 2011). The algorithm combines epipolar deviations (Swaminathan et al. in Lect Notes Comput Sci 2350:508-523, 2002) and appearance distortion, and succeeds in localizing specular objects in computer-rendered and real scenes, across a wide range of camera motions and speeds, object sizes and shapes, and performs well under image noise and blur conditions. © 2014 Springer-Verlag Berlin Heidelberg

Shatter and splatter: The contribution of mechanical and optical properties to the perception of soft and hard breaking materials

“…Recent studies have shown that image motion can also greatly affect the perception of surface optical properties Marlow & Anderson, 2016;Oren & Nayar, 1997;Sakano & Ando, 2010;Wendt, Faul, Ekroll, & Mausfeld, 2010;Yilmaz & Doerschner, 2014). For example, identified three motion cues (coverage, divergence, and 3-D shape reliability) that the visual system could potentially use to distinguish between moving shiny and textured matte surfaces that otherwise looked identical when presented as static images.…”

Seeing jelly

“…It has been already shown that image motion cues are important to recognize materials. For example, optical flow patterns are used to discriminate matte from glossy surfaces [Doerschner et al 2011;Yilmaz and Doerschner 2014]. Moreover, the pattern of image motion speed can be utilized to discriminate the rigid from non-rigid cylinders [Jain and Zaidi 2011].…”

‘Proto-rivalry’: how the binocular brain identifies gloss

“…working out the disparities for which the two eyes saw the same part of the surrounding environment [ 15 ]). Using these matches, we measured two properties of the binocular vector fields produced by different vIPs ( figure 2 e ): (i) the distributions of horizontal disparity gradients and (ii) the magnitude of vertical disparities (analogous to epipolar deviations in optic flow fields created by specular surfaces, [ 17 , 18 ]). We previously suggested that these binocular measurements may provide important cues to indicate that some depth signals created by specular surfaces are intrinsically unreliable [ 13 ].…”