Adversarial Monte Carlo denoising with conditioned auxiliary feature modulation

Abstract

including regression-based and learning-based methods, have been explored to achieve better rendering quality with less computational cost. However, most of these methods rely on handcrafted optimization objectives, which lead to artifacts such as blurs and unfaithful details. In this paper, we present an adversarial approach for denoising Monte Carlo rendering. Our key insight is that generative adversarial networks can help denoiser networks to produce more realistic high-frequency details and global illumination by learning the distribution from a set of high-quality Monte Carlo path tracing images. We also adapt a novel feature modulation method to utilize auxiliary features better, including normal, albedo and depth. Compared to previous state-of-the-art methods, our approach produces a better reconstruction of the Monte Carlo integral from a few samples, performs more robustly at different sample rates, and takes only a second for megapixel images.

Unsupervised Image Reconstruction for Gradient‐Domain Volumetric Rendering

“…Lin et al [LWWH20] introduced light transport covariance from the light source to improve high‐frequency lighting details. A generative adversarial network (GAN) MC denoising method was proposed by Xu et al [XZW19] to achieve higher perceptual quality. Gharbi et al [GLA∗19] proposed a sample‐based denoising method (SBMC), by splatting each sample onto nearby pixels to produce denoised results which allows very low sampling rate.…”

DMCR-GAN: Adversarial Denoising for Monte Carlo Renderings with Residual Attention Networks and Hierarchical Features Modulation of Auxiliary Buffers

“…Learning-based Monte Carlo denosing task learns a mapping G(•) to translate the noisy input I n to its noise-free form O nf with the help of auxiliary buffers F . In this paper, we decompose the noisy input into diffuse and specular components and train them separately [Xu et al 2019]. They share the same network architecture but different parameters.…”

“…All training data are cropped into 128x128 patches by importance sampling [Bako et al 2017]. Moreover, similar to [Bako et al 2017;Xu et al 2019], we apply a log transform to compress the high dynamic range (HDR) of color values and use untextured diffuse component by dividing albedo buffer and normalize auxiliary buffers to the same range [0.0-1.0]. Training setup.…”

“…[Wong and Wong 2019] used several residual blocks to directly inference the noise-free images. [Xu et al 2019] proposed a modulation method that element-wise scaling and biasing noisy input with auxiliary buffers via generative adversarial network (GAN) [Goodfellow et al 2014]. However, there are mainly two problems when denosing Monte Carlo rendered image which are as follows.…”

“…However, there are mainly two problems when denosing Monte Carlo rendered image which are as follows. First, [Bako et al 2017] and [Vogels et al 2018] simply blended noisy input with auxiliary buffers as the network input or extracted shallow features of auxiliary buffers to modulate the noisy input [Xu et al 2019] in each residual block, which cannot make full use of abundant low-frequency and high-frequency information of auxiliary buffers. Second, all channel features and spatial features are treated equally.…”