Neural BRDF Representation and Importance Sampling

Abstract

Controlled capture of real-world material appearance yields tabulated sets of highly realistic reflectance data. In practice, however, its high memory footprint requires compressing into a representation that can be used efficiently in rendering while remaining faithful to the original. Previous works in appearance encoding often prioritized one of these requirements at the expense of the other, by either applying high-fidelity array compression strategies not suited for efficient queries during rendering, or by fitting a compact analytic model that lacks expressiveness. We present a compact neural network-based representation of BRDF data that combines high-accuracy reconstruction with efficient practical rendering via built-in interpolation of reflectance. We encode BRDFs as lightweight networks, and propose a training scheme with adaptive angular sampling, critical for the accurate reconstruction of specular highlights. Additionally, we propose a novel approach to make our representation amenable to importance sampling: rather than inverting the trained networks, we learn to encode them in a more compact embedding that can be mapped to parameters of an analytic BRDF for which importance sampling is known. We evaluate encoding results on isotropic and anisotropic BRDFs from multiple real-world datasets, and importance sampling performance for isotropic BRDFs mapped to two different analytic models.

Neural BRDFs: Representation and Operations

“…In Table 1, we compare our method against other three related works, regarding the representation accuracy for specularity, ability for representing SVBRDFs, and the generality of the model. Regarding representation accuracy, both Sztrajman et al [2021] and our method are able to handle sharp specular BRDFs, while Rainer et al [2019] and Rainer et al [2020] are less accurate. However, Sztrajman et al [2021] require a decoder for each BRDF representation, which makes it not applicable for SVBRDF representation.…”

“…Regarding representation accuracy, both Sztrajman et al [2021] and our method are able to handle sharp specular BRDFs, while Rainer et al [2019] and Rainer et al [2020] are less accurate. However, Sztrajman et al [2021] require a decoder for each BRDF representation, which makes it not applicable for SVBRDF representation. Regarding the generality, Sztrajman et al [2021] require training for each BRDF, and Rainer et al [2019] require training for each single SVBRDF.…”

“…In their method, one neural network is trained per material. More recently, Sztrajman et al [2021] represent each BRDF with a decoder structure, resulting in better quality, at the cost of more storage for each BRDF. We categorize the aforementioned approaches as specialized methods, because one neural network only represents one material/BRDF in these works.…”

“…Being a generalized method, our method achieves comparable representation quality as compared to specialized methods, such as Sztrajman et al [2021], but it requires much less storage, making it more suitable for SVBRDFs' representation. More importantly, our method not only represents BRDFs with latent vectors, but also supports a rich set of operations on the latent vectors.…”

“…The representation should be general-purpose, taking in any BRDF as input, outputting its corresponding latent vector. In other words, it is not sufficient to train one different network for each BRDF [Sztrajman et al 2021], or even one network per SVBRDF [Rainer et al 2019].…”