Shape from shading in the light of mutual illumination

Abstract

It has been argued for many years that human subjects recover three dimensional shape cues from the distribution of shading on an object. Many machine vision researchers have interpreted this skill as an indication that the distribution of radiance in regions of an object that do not lie in shadow, can be integrated to yield a dense depth map. Unfortunately, this belief is critically dependent on an extremely simple photometric model, known as the image irradiance equation (see [11] for a clear exposition of this approach).This model is flawed because it assumes that radiance is a function of a purely local geometric property, the surface normal. It ignores the fact that patches of surface reflect light not only to an imaging sensor, but also to other patches of surface (an effect known as "mutual illumination"), making the distribution of radiance a complicated function of the global scene geometry.Unfortunately, it is easy to observe the effects of this redistribution in simple experiments [18,4,16,12,13]. We show some examples in figures 1-3. At first glance, the effects of mutual illumination appear rich in desirable cues to shape and to absolute surface lightness. However, the mathematical complexity of mutual illumination effects means that it is very difficult to see how these cues may be exploited. On the other hand, there is reason to believe that humans can exploit these cues to some extent [6,7].The purpose of this article is to explore the implications of mutual illumination and to consider what shading cues can in fact be used to recover shape information. We show that although radiance itself is not a reliable shape cue, discontinuities in radiance are, as they can appear only at distinguished points on surfaces. Furthermore, the events that cause discontinuities in radiance are geometrically simple.

A novel algorithm for color constancy

“…Given that a feasible set of surface colors can be achieved at a very early stage of visual processing, it seems reasonable to maintain this representation until the last possible moment. If we do this, other sources of visual information may provide cues that reduce the size of the feasible set--shape and surface color, for example, are deeply entwined by mutual illumination effects (Gershon et al 1986;Forsyth and Zisserman 1989;Forsyth and Zisserman 1990;Ho et al 1989). Crule will produce color descriptors for two color images, of the kind used by, for example Land (Land 1959a;Land 1959b).…”

Learning to Use Illumination Gradients as an Unambiguous Cue to Three Dimensional Shape

“…The gradients in the roof stimuli are due to intensity drop-off with distance from the light source, while the ones in the corner stimuli are also influenced by inter-reflections between surfaces and shading. Our visual system successfully assigns approximately correct shape in both situations, lending credence to sophisticated shape from shading algorithms that have shown that inter-reflections are in fact a possible source of useful 3-D information [37], [38] over traditional algorithms [15] where inter-reflections/mutual illuminations are ignored and lead to incorrect shape estimates.…”

Variable-Source Shading Analysis

“…The effects of other surfaces can be significant. In practical observations of real geometries, Forsyth and Zisserman demonstrated substantial differences between radiosity predicted by local shading models and those observed in practice; the differences could be explained by interreflections (Forsyth and Zisserman 1989, 1990, 1991. Interreflected shading is pronounced in concave regions, and can result in a significant increase in surface radiosity at the brightest points and in large qualitative changes in the shape of the surface intensity profile (see figures in Forsyth and Zisserman 1991 for examples).…”