The Episolar Constraint: Monocular Shape from Shadow Correspondence

Abstract

Shadows encode a powerful geometric cue: if one pixel casts a shadow onto another, then the two pixels are colinear with the lighting direction. Given many images over many lighting directions, this constraint can be leveraged to recover the depth of a scene from a single viewpoint. For outdoor scenes with solar illumination, we term this the episolar constraint, which provides a convex optimization to solve for the sparse depth of a scene from shadow correspondences, a method to reduce the search space when finding shadow correspondences, and a method to geometrically calibrate a camera using shadow constraints. Our method constructs a dense network of nonlocal constraints which complements recent work on outdoor photometric stereo and cloud based cues for 3D. We demonstrate results across a variety of time-lapse sequences from webcams "in the wild."

One-day outdoor photometric stereo via skylight estimation

“…We detect the shadowed pixels explicitly using the shadow estimation method for [2]. This method is an expectation maximization approach which simultaneously estimates shadows, albedo, surface normals, and ambient light.…”

“…7 shows the albedo and surface normal estimation of our method using one-day images of Düsseldorf and Meersburg datasets. The shadow detection method for [2] which also estimates albedo, surface normals, and ambient light is compared using the same one day images. Abrams et al [1]'s results using 500 images of many months are obtained from their original paper.…”

Scene geometry from several partly cloudy days

“…Such approaches require clear sky conditions, or a method for filtering out cloudy conditions. Unlike our approach, these methods are currently restricted to estimating surface normals [1], [9], which would be challenging to convert into depth estimates in a general outdoor environment, or only provide depth estimates around shadow casters [2]. An additional limitation of all of these approaches is that the scale of the scene is ambiguous.…”

“…These approaches all rely on strong assumptions about the scene, including clear sky conditions [1], [2], [9] or constant albedo [1], [9]. The most commonly used cues are based on appearance changes due to sun motion, such as photometric changes [1], [9] and cast shadows geometry [2].…”

“…Several cues and approaches have been proposed for estimating shape information from video of an outdoor scene captured by a static camera [1], [2], [6], [9], [11]. These approaches all rely on strong assumptions about the scene, including clear sky conditions [1], [2], [9] or constant albedo [1], [9].…”