Dense shape reconstruction of a moving object under arbitrary, unknown lighting

Simakov,; Фролова,; Basri, Basri

doi:10.1109/iccv.2003.1238628

Cited by 52 publications

(20 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Within stereo, Simakov et al propose a dense correspondence measure for Lambertian surfaces that accounts for first order spherical harmonics of environment illumination [16]. Similar to our setup, albeit limited to Lambertian reflectance and directional lighting, passive photometric stereo methods [9,18] use object motion to reconstruct a dense depth map.…”

Section: Related Workmentioning

confidence: 99%

What Object Motion Reveals about Shape with Unknown BRDF and Lighting

Chandraker

Reddy

Wang

et al. 2013

2013 IEEE Conference on Computer Vision and Pattern Recognition

View full text Add to dashboard Cite

We present a theory that addresses the problem of determining shape from the (small or differential) motion of an object with unknown isotropic reflectance, under arbitrary unknown distant illumination, for both orthographic and perpsective projection. Our theory imposes fundamental limits on the hardness of surface reconstruction, independent of the method involved. Under orthographic projection, we prove that three differential motions suffice to yield an invariant that relates shape to image derivatives, regardless of BRDF and illumination. Under perspective projection, we show that four differential motions suffice to yield depth and a linear constraint on the surface gradient, with unknown BRDF and lighting. Further, we delineate the topological classes up to which reconstruction may be achieved using the invariants. Finally, we derive a general stratification that relates hardness of shape recovery to scene complexity. Qualitatively, our invariants are homogeneous partial differential equations for simple lighting and inhomogeneous for complex illumination. Quantitatively, our framework shows that the minimal number of motions required to resolve shape is greater for more complex scenes. Prior works that assume brightness constancy, Lambertian BRDF or a known directional light source follow as special cases of our stratification. We illustrate with synthetic and real data how potential reconstruction methods may exploit our framework.

show abstract

Section: Related Workmentioning

confidence: 99%

What Object Motion Reveals about Shape with Unknown BRDF and Lighting

Chandraker

Reddy

Wang

et al. 2013

2013 IEEE Conference on Computer Vision and Pattern Recognition

View full text Add to dashboard Cite

show abstract

“…This result often enables computer vision algorithms, previously restricted to point sources without attached shadows, to work in general complex lighting. There has been considerable work on novel algorithms for lighting-insensitive recognition, photometric stereo, and relighting [2], [1], [7], [22], [28], [26]. In graphics, the general convolution formulas have been used for rendering with environment maps [20], and insights have been widely adopted for forward and inverse rendering (for example, [19], [23]).…”

Section: Introductionmentioning

confidence: 99%

A Theory Of Frequency Domain Invariants: Spherical Harmonic Identities for BRDF/Lighting Transfer and Image Consistency

Mahajan

Ramamoorthi

Curless

2008

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Abstract-This paper develops a theory of frequency domain invariants in computer vision. We derive novel identities using spherical harmonics, which are the angular frequency domain analog to common spatial domain invariants such as reflectance ratios. These invariants are derived from the spherical harmonic convolution framework for reflection from a curved surface. Our identities apply in a number of canonical cases, including single and multiple images of objects under the same and different lighting conditions. One important case we consider is two different glossy objects in two different lighting environments. For this case, we derive a novel identity, independent of the specific lighting configurations or BRDFs, that allows us to directly estimate the fourth image if the other three are available. The identity can also be used as an invariant to detect tampering in the images. Although this paper is primarily theoretical, it has the potential to lay the mathematical foundations for two important practical applications. First, we can develop more general algorithms for inverse rendering problems, which can directly relight and change material properties by transferring the BRDF or lighting from another object or illumination. Second, we can check the consistency of an image to detect tampering or image splicing.

show abstract

“…Although the availability of the constraint was limited in principle to Lambertian surface as is the case with the other sophisticated approaches [11,13], the thrust of this paper is to extend it to the situation that the object surface partly takes on specular reflection. In the case of stereo, in the presence of specular reflection, it has been shown possible to determine trinocular configurations such that at least one stereo pair can provide correct depth estimate at each scene point visible to all cameras [3].…”

Section: Introductionmentioning

confidence: 99%

3D Surface Reconstruction of a Moving Object in the Presence of Specular Reflection

Maki

2005

Image Analysis and Processing – ICIAP 2005

View full text Add to dashboard Cite

Abstract. We present a new scheme for 3D surface reconstruction of a moving object in the presence of specular reflection. We basically search for the depth at each point on the surface of the object while exploiting the recently proposed geotensity constraint [7] that accurately governs the relationship between four or more images of a moving object in spite of the illumination variance due to object motion. The thrust of this paper is then to extend the availability of the geotensity constraint to the case that specularities are also present. The key idea is to utilise the fact that highlights shift on the surface due to object motion. I.e., we employ five or more images as inputs, and interchangeably utilise a certain intensity subset consisting of four projected intensities which is the least influenced by the specular component. We illustrate the relevancy of our simple algorithm also through experiments.

show abstract

Dense shape reconstruction of a moving object under arbitrary, unknown lighting

Cited by 52 publications

References 15 publications

What Object Motion Reveals about Shape with Unknown BRDF and Lighting

What Object Motion Reveals about Shape with Unknown BRDF and Lighting

A Theory Of Frequency Domain Invariants: Spherical Harmonic Identities for BRDF/Lighting Transfer and Image Consistency

3D Surface Reconstruction of a Moving Object in the Presence of Specular Reflection

Contact Info

Product

Resources

About