A Dual Theory of Inverse and Forward Light Transport

Abstract

A cornerstone of computer graphics is the solution of the rendering equation for interreflections, which allows the simulation of global illumination, given direct lighting or corresponding light source emissions. This paper lays the foundations for the inverse problem, whereby a dual theoretical framework is presented for inverting the rendering equation to undo interreflections in a real scene, thereby obtaining the direct lighting. Inverse light transport is of growing importance, enabling a variety of new applications like separation of individual bounces of the light transport, and projector radiometric compensation to display images free of global illumination artifacts in real-world environments that exhibit complex geometric and reflectance properties. However, solving the inverse problem involves the inversion of a large light transport matrix (acquired by measurement on real scenes). While straightforward matrix inversion is intractable for most realistic resolutions, there is scant prior work on either theoretical foundations or fast computational algorithms to meet the objectives of inverse light transport.In this paper, we develop a mathematical theory that exposes the duality of forward and inverse light transport. Forward rendering also formally involves a matrix or operator inversion, which is conceptually equivalent to a multi-bounce Neumann series expansion. We show the existence of an analogous series for the inverse problem. However, the convergence is oscillatory in the inverse case, with more interesting conditions on material reflectance. Importantly, we give physical meaning to this duality, by showing that each term of our inverse series cancels an interreflection bounce, just as the forward series adds them.In algorithmic terms, we develop the analog of iterative finite element methods like forward radiosity to efficiently solve light transport inversion. Our iterative inverse light transport algorithm is very fast, requiring only matrix-vector multiplications, and follows directly from the dual theoretical formulation. We also explore the connections to forward rendering in terms of Monte Carlo and wavelet-based techniques. As an initial practical application, we first acquire the light transport of a real static scene, and then demonstrate iterative inversion for radiometric compensation on high-resolution datasets, as well as rapid separation of the bounces of global illumination. *

From the Rendering Equation to Stratified Light Transport Inversion

“…This derivation makes explicit the connection between the iterative computation framework of Bimber et al (2006) and Bimber (2006) with known scene geometry and the stratified inversion framework of Ng et al (2009) with only light transport measurement as input. Second, we show a Neumann interpretation for the stratified inverses in terms of physical bounces of light, which brings out an interesting correspondence between the forward and the inverse light transport (a brief summary of this result is described by us in Bai et al (2010), but this paper presents the full derivation and analysis.) Although Seitz et al (2005) has showed that inverse light transport can be used for separating light bounces in forward light transport, the physical meaning of the polynomial terms in inverse light transport is novel.…”

“…This is done using Jacobi iteration on the projector input vector, with the form factor matrix derived from the scene geometry. In the case of unknown scene geometry, but given the light transport of the scene, we showed in Bai et al (2010) that projector radiometric compensation can be similarly computed iteratively. These approaches relate closely to the radiosity method for diffuse global illumination in forward rendering (Hanrahan et al, 1991;Gortler et al, 1993).…”

Compressive Inverse Light Transport

“…The first approach solves Eq. 1 as a system of linear equations [1]. Efficient methods of this approach are the Jacobi method and the Gauss-Seidel method that involves iterative vector-matrix multiplication.…”

“…It is common that f-LTM T under focused light sources such as light pixels of a projector are diagonally dominant 1 [1,8,23]. An example of such f-LTM is shown in Fig.1(a) with another one shown in Section 3 of the supplementary.…”

“…Hence, (T −1 ) (p) is also compressible. Convergence of the Neumann series for forward and inverse light transport can be found in [1,16] . Now we concluded thatT −1 is sparse which means that T −1 must be sparse, and hence T −1 can be sparsely represented but not as sparse as the sparse representation of T. This conclusion is demonstrated in Fig.1.…”