Inverse shade trees for non-parametric material representation and editing

Abstract

Recent progress in the measurement of surface reflectance has created a demand for non-parametric appearance representations that are accurate, compact, and easy to use for rendering. Another crucial goal, which has so far received little attention, is editability: for practical use, we must be able to change both the directional and spatial behavior of surface reflectance (e.g., making one material shinier, another more anisotropic, and changing the spatial "texture maps" indicating where each material appears). We introduce an Inverse Shade Tree framework that provides a general approach to estimating the "leaves" of a user-specified shade tree from highdimensional measured datasets of appearance. These leaves are sampled 1-and 2-dimensional functions that capture both the directional behavior of individual materials and their spatial mixing patterns. In order to compute these shade trees automatically, we map the problem to matrix factorization and introduce a flexible new algorithm that allows for constraints such as non-negativity, sparsity, and energy conservation. Although we cannot infer every type of shade tree, we demonstrate the ability to reduce multigigabyte measured datasets of the Spatially-Varying Bidirectional Reflectance Distribution Function (SVBRDF) into a compact representation that may be edited in real time.

Robust estimation of surface properties and interpolation of shadow/specularity components

“…Appearance-based rendering approaches (e.g., [27,28,29]) are indeed concerned with appearance and BRDF modelling, and not surface recovery. Here we strike a balance by proposing a hierarchy of methods that progressively trade speed for accuracy of recovered surface properties.…”

“…For the intermediate-sized image sets used here (as opposed to a sparse set aimed at in Reflectance Sharing) the results displayed below justify the suitability of this assumption. The method set out here is considerably simpler than many methods advanced for strictly appearance modelling (e.g., [27,28,29]) while at the same time also recovering surface properties such as surface normal.…”

Photometric stereo with non-parametric and spatially-varying reflectance

“…The most prominent of these approaches is work by Lawrence et al [12] who solve the factorization problem using alternating constrained least squares. Their algorithm is again based on the assumption that spatially varying reflectance can be represented as a weighted sum of a small set of materials.…”

“…Second, we utilize a non-parametric bi-variate approximation of the BRDF. Finally, we assume that surfaces are composed of a small number of "basis" materials and solve a factorization problem similar to that of Lawrence et al [12], but tailored to our differing setup (single viewpoint, recovery of surface geometry, fewer image measurements).…”

“…If W = (w 1 , ..., w n ) ⊤ and B are unknown, then we can estimate them using the method of alternating constrained least squares (ACLS), as described by Lawrence et al [12]. ACLS works by alternately updating W and B to minimize the residual between measured intensities and predicted intensities.…”