DMCR-GAN: Adversarial Denoising for Monte Carlo Renderings with Residual Attention Networks and Hierarchical Features Modulation of Auxiliary Buffers

Abstract

Figure 1: We propose an adversarial approach for denoising Monte Carlo Renderings (DMCR-GAN). Our network uses several convolution dense blocks to extract rich information of auxiliary buffers and then use these different hierarchical features to modulate the noisy features in the residual blocks. Moreover, we introduce the channel and spatial attention mechanism to exploit the dependencies of inter-channel and inter-spatial features. We use the public dataset [Bitterli 2016] rendered by Tungsten renderer and compare our results with works of NFOR [Bitterli et al. 2016], KPCN[Bako et al. 2017] and ACFM [Xu et al. 2019] methods. The noisy input image and the ground truth image are rendered with 32 spp and 32k spp respectively. The values above images indicate the RMSE, PSNR and SSIM metrics. The comparison of denoised images generated by our proposed and other methods is shown in the supplementary material.

FoVolNet: Fast Volume Rendering using Foveated Deep Neural Networks

“…Wong et al [62] introduce residual connections for direct prediction networks to improve single-frame image quality. To the same end, Xu et al [64] and Lu et al [32] conducted experiments on using adversarial networks [11] to train direct prediction models. More recently, Hofmann et al [17] have applied direct prediction to the domain of volume path tracing.…”

“…Extensions often include skip connections and recurrent feedback, which tend to increase image quality and temporal stability. Other works [2,32,56,62,64] use more conventional CNN or RNN models. Interestingly, there is no apparent connection between chosen architecture (U-Net, CNN, RNN) and the denoising approach (direct prediction, kernel prediction).…”

“…Interestingly, there is no apparent connection between chosen architecture (U-Net, CNN, RNN) and the denoising approach (direct prediction, kernel prediction). Several works [17,32,64] also use an additional critique network for their adversarial training. Named after its shape, the W-Net poses an extension to the U-Net and was introduced by Thomas et al [55].…”

DGAN-KPN: Deep Generative Adversarial Network and Kernel Prediction Network for Denoising MC Renderings

“…Again in 2020, Yifan et al [19] proposed the Adversarial Denoising for MC Renderings network, which used many convoluted dense blocks to extract rich information of auxiliary buffers, and then used these various hierarchical features to modify the noisy features in the residual blocks. Furthermore, they presented the channel mechanism and spatial interest to exploit property dependencies between channels and spatial features.…”

A denoising model for MC rendered images based on fusion kernel prediction and generation of adversarial networks

“…Researchers Xu et al [6] and Lu et al [7] used Generative Adversarial Networks (GANs) for offline denoising of images created by Monte Carlo rendering. Their methods outperform classic Convolutional Neural Network (CNN) [8] approaches in terms of denoising.…”