Unsupervised Image Reconstruction for Gradient‐Domain Volumetric Rendering

Abstract

Gradient-domain rendering can highly improve the convergence of light transport simulation using the smoothness in image space. These methods generate image gradients and solve an image reconstruction problem with rendered image and the gradient images. Recently, a previous work proposed a gradient-domain volumetric photon density estimation for homogeneous participating media. However, the image reconstruction relies on traditional L1 reconstruction, which leads to obvious artifacts when only a few rendering passes are performed. Deep learning based reconstruction methods have been exploited for surface rendering, but they are not suitable for volume density estimation. In this paper, we propose an unsupervised neural network for image reconstruction of gradient-domain volumetric photon density estimation, more specifically for volumetric photon mapping, using a variant of GradNet with an encoded shift connection and a separated auxiliary feature branch, which includes volume based auxiliary features such as transmittance and photon density. Our network smooths the images on global scale and preserves the high frequency details on a small scale. We demonstrate that our network produces a higher quality result, compared to previous work. Although we only considered volumetric photon mapping, it's straightforward to extend our method for other forms, like beam radiance estimation. CCS Concepts • Computing methodologies → Neural network; Ray tracing; † Joint first author.

Automatic Feature Selection for Denoising Volumetric Renderings

“…Subsequently, a pipeline for interactive volume rendering was proposed [HHCM21] which includes a lightweight neural denoiser that uses a different set of auxiliary features. In the context of gradient‐domain volumetric photon density estimation, Xu et al [XSW∗20] proposed a method using auxiliary volume‐specific feature buffers like transmittance and photon density to denoise global homogeneous volumes. Among the more traditional methods, Iglesias‐Guitian et al [IGMM20] propose a real‐time denoising pipeline for volumetric renderings using a history buffer.…”

“…There are established sets of features that are commonly used to improve the denoising quality of surfaces [RMZ13, VRM∗18]. However, while various auxiliary features for denoising volumetric content have been proposed in the past [HMES20, XSW∗20, HHCM21], there is no established feature set for volume denoising.…”

A survey on deep learning-based Monte Carlo denoising

“…Xu et al [64] jointly leveraged gradient-domain information and photon mapping techniques for rendering homogenous participating media. They adopt an unsupervised gradient-domain deep learning framework [47] for image reconstruction of gradientdomain volumetric photon density estimation.…”

A survey on rendering homogeneous participating media

“…With gradient domain rendering, smoother results are obtained than the original method. More recently, a deep learning based reconstruction approach for participating media has been proposed [53], yielding a better reconstruction solution than Poisson reconstruction. We will describe this approach in Section 9.…”

“…Xu et al [53] proposed an unsupervised neural network for image reconstruction of gradientdomain volumetric photon density estimation, more specifically for volumetric photon mapping, using a variant of GradNet [67] with an encoded shift connection and a separated auxiliary feature branch; it includes volume based auxiliary features such as transmittance and photon density. This network smooths images at a global scale and preserves highfrequency details on a small scale.…”