A Theory Of Frequency Domain Invariants: Spherical Harmonic Identities for BRDF/Lighting Transfer and Image Consistency

Abstract

Abstract-This paper develops a theory of frequency domain invariants in computer vision. We derive novel identities using spherical harmonics, which are the angular frequency domain analog to common spatial domain invariants such as reflectance ratios. These invariants are derived from the spherical harmonic convolution framework for reflection from a curved surface. Our identities apply in a number of canonical cases, including single and multiple images of objects under the same and different lighting conditions. One important case we consider is two different glossy objects in two different lighting environments. For this case, we derive a novel identity, independent of the specific lighting configurations or BRDFs, that allows us to directly estimate the fourth image if the other three are available. The identity can also be used as an invariant to detect tampering in the images. Although this paper is primarily theoretical, it has the potential to lay the mathematical foundations for two important practical applications. First, we can develop more general algorithms for inverse rendering problems, which can directly relight and change material properties by transferring the BRDF or lighting from another object or illumination. Second, we can check the consistency of an image to detect tampering or image splicing.

Generalizing the mean intercept length tensor for gray-level images

“…In the literature, t is usually referred to as the half-cosine function. [32][33][34] It is important to remark that in general C(v) = C(−v). enough value of is used, this result can be generalized to any shape of the interface between phases in 2D as…”

Estimating all frequency lighting using a color/depth image

“…In general, existing approaches in inverse lighting can be divided into two classes based upon their representation of lighting; basis representation and point representation. The basis representation has been widely adopted for recovering the low frequency lighting [2], [3], [4], [5]. Nimeroff et al [6] first used a linear combination of steerable basis functions to represent the lighting.…”

“…Later, Ramamoorthi [7] has shown that the spherical harmonics basis function (SH) is an optimal basis function to represent the illumination of a convex Lambertian object. Most recently, Mahajan et al [5] employed the image restoration framework into inverse lighting. They used a wiener filter to robustly recover the low frequency lighting as well as to infer the high frequency lighting from the low frequency lighting.…”

“…For estimating all frequency lighting, the proposed approach suggests a new representation, combining the basis representation and the point representation. Although previous methods [4], [5] have attempted to achieve all frequency lighting using the SH basis representation, the resultant lighting presented the unpleasant noises, e.g. ringing effects.…”

Detecting contrast enhancement based image forgeries by parallel approach

“…2. Content changing operations includes splicing and composition [17,18]. Contrast enhancement including statistical intrinsic fingerprints [7,16], Gamma correction [8], reverse engineering of double Jpeg compressed images [9], noise features [15].…”