Adaptive Incident Radiance Field Sampling and Reconstruction Using Deep Reinforcement Learning

Abstract

Serious noise affects the rendering of global illumination using Monte Carlo (MC) path tracing when insufficient samples are used. The two common solutions to this problem are filtering noisy inputs to generate smooth but biased results and sampling the MC integrand with a carefully crafted probability distribution function (PDF) to produce unbiased results. Both solutions benefit from an efficient incident radiance field sampling and reconstruction algorithm. This study proposes a method for training quality and reconstruction networks (Q- and R-networks, respectively) with a massive offline dataset for the adaptive sampling and reconstruction of first-bounce incident radiance fields. The convolutional neural network (CNN)-based R-network reconstructs the incident radiance field in a 4D space, whereas the deep reinforcement learning (DRL)-based Q-network predicts and guides the adaptive sampling process. The approach is verified by comparing it with state-of-the-art unbiased path guiding methods and filtering methods. Results demonstrate improvements for unbiased path guiding and competitive performance in biased applications, including filtering and irradiance caching.

A survey on deep learning-based Monte Carlo denoising

“…Instead of using light-field-space features for imagespace denoising, another category of research aims to directly reconstruct the denoised incident radiance field, i.e., the local light field at each pixel, for advanced goals such as unbiased path guiding [40][41][42]. We cover such works in Section 5.4.…”

“…As reference, the trained network takes a small number of uniform initial samples as input to predict the full incident radiance field of each pixel, which is used to guide the remaining samples to generate the final results. Instead of single-pass path guiding, another method takes a progressive adaptive sampling strategy that iteratively uses last-iteration samples to guide the sampling process for the next iteration [41]. In order to guide the progressive sampling process, the method considers the sampling as an action that can produce rewards, i.e., reducing reconstruction errors, and trains a quality-predicting neural network to predict the gain of different actions in a deep reinforcement learning (DRL) way [68,69].…”

“…Fig. 9 Equal-time comparisons between a biased MC denoiser (Bako et al [16]) and unbiased path guiding using deep learning to generate guidance (Huo et al [41]). Results are rendered within 1, 2, 8, 30, 60, and 120 minutes from top to bottom.…”

Hierarchical neural reconstruction for path guiding using hybrid path and photon samples

“…While images are convenient for neural networks, they consume more memory when detailed light distributions are needed. Huo et al [2020] used a reinforcement learning technique to guide the samples, but their method is also limited to the first bounce. Zhu et al [2020b] used photons as the primary source to estimate the local light distributions, and use them to guide all bounces.…”

Hash-Based Hierarchical Caching and Layered Filtering for Interactive Previews in Global Illumination Rendering

“…Their view-dependent radiance caching works directly with outgoing radiance at surfaces instead of incoming radiance distributions. In a recent paper, Huo et al [48] propose the use of quality and reconstruction networks with a large offline dataset for the adaptive sampling and reconstruction of first-bounce incident radiance fields. The reconstruction network is based on a convolutional neural network.…”