Kernel-predicting convolutional networks for denoising Monte Carlo renderings

227

248

High dynamic range (HDR) imaging provides the capability of handling real world lighting as opposed to the traditional low dynamic range (LDR) which struggles to accurately represent images with higher dynamic range. However, most imaging content is still available only in LDR. This paper presents a method for generating HDR content from LDR content based on deep Convolutional Neural Networks (CNNs) termed ExpandNet. ExpandNet accepts LDR images as input and generates images with an expanded range in an end‐to‐end fashion. The model attempts to reconstruct missing information that was lost from the original signal due to quantization, clipping, tone mapping or gamma correction. The added information is reconstructed from learned features, as the network is trained in a supervised fashion using a dataset of HDR images. The approach is fully automatic and data driven; it does not require any heuristics or human expertise. ExpandNet uses a multiscale architecture which avoids the use of upsampling layers to improve image quality. The method performs well compared to expansion/inverse tone mapping operators quantitatively on multiple metrics, even for badly exposed inputs.

Section: Related Workmentioning

confidence: 99%

ExpandNet: A Deep Convolutional Neural Network for High Dynamic Range Expansion from Low Dynamic Range Content

Marnerides

Bashford-Rogers

Hatchett

et al. 2018

227

248

“…Of all the variance reduction techniques proposed over the years, MC denoising [SD11, SD12, RMZ13, KBS15], in particular, has helped to fuel the recent, rapid adoption of path tracing. MC denoisers for both high‐end production [BVM∗ 17, VRM∗ 18] and real‐time [CKS∗ 17, SKW∗ 17, SPD18] rendering systems have demonstrated impressive results at low sampling rates for their respective applications (16 samples/pixel for production, 1‐2 samples/pixel for real‐time). Still, their results could substantially improve if they could be provided input images that are more converged than the ones they usually operate on.…”

Section: Introductionmentioning

confidence: 99%

Offline Deep Importance Sampling for Monte Carlo Path Tracing

Bako

Meyer²,

DeRose³

et al. 2019

Self Cite

Although modern path tracers are successfully being applied to many rendering applications, there is considerable interest to push them towards ever‐decreasing sampling rates. As the sampling rate is substantially reduced, however, even Monte Carlo (MC) denoisers–which have been very successful at removing large amounts of noise–typically do not produce acceptable final results. As an orthogonal approach to this, we believe that good importance sampling of paths is critical for producing better‐converged, path‐traced images at low sample counts that can then, for example, be more effectively denoised. However, most recent importance‐sampling techniques for guiding path tracing (an area known as “path guiding”) involve expensive online (per‐scene) training and offer benefits only at high sample counts. In this paper, we propose an offline, scene‐independent deep‐learning approach that can importance sample first‐bounce light paths for general scenes without the need of the costly online training, and can start guiding path sampling with as little as 1 sample per pixel. Instead of learning to “overfit” to the sampling distribution of a specific scene like most previous work, our data‐driven approach is trained a priori on a set of training scenes on how to use a local neighborhood of samples with additional feature information to reconstruct the full incident radiance at a point in the scene, which enables first‐bounce importance sampling for new test scenes. Our solution is easy to integrate into existing rendering pipelines without the need for retraining, as we demonstrate by incorporating it into both the Blender/Cycles and Mitsuba path tracers. Finally, we show how our offline, deep importance sampler (ODIS) increases convergence at low sample counts and improves the results of an off‐the‐shelf denoiser relative to other state‐of‐the‐art sampling techniques.

“…Jia et al [JDTG16] proposed to predict a dynamic convolution layer or a dynamic local filtering layer from input data and apply the layer on the input. The latter strategy, namely kernel prediction method, has been used for denoising bursts of images [MBC*18] and denoising Monte Carlo renderings [BVM*17; VRM*18]. Deformable convolutional network [DQX*17] predicts the sampling locations of the convolution operator and it was further improved by modulating the input feature amplitudes from different spatial locations [ZHLD18].…”

Section: Related Workmentioning

confidence: 99%

“…(1) and learn the kernel weights w (·,·) from a CNN model. Our strategy is similar to the kernel prediction method [JDTG16], which has been applied to denoising [MBC*18; BVM*17; VRM*18]. However, directly applying Equ.…”

Section: Introductionmentioning

confidence: 99%

Learning Explicit Smoothing Kernels for Joint Image Filtering

Fang

Wang

Shamir

et al. 2019

Smoothing noises while preserving strong edges in images is an important problem in image processing. Image smoothing filters can be either explicit (based on local weighted average) or implicit (based on global optimization). Implicit methods are usually time‐consuming and cannot be applied to joint image filtering tasks, i.e., leveraging the structural information of a guidance image to filter a target image. Previous deep learning based image smoothing filters are all implicit and unavailable for joint filtering. In this paper, we propose to learn explicit guidance feature maps as well as offset maps from the guidance image and smoothing parameter that can be utilized to smooth the input itself or to filter images in other target domains. We design a deep convolutional neural network consisting of a fully‐convolution block for guidance and offset maps extraction together with a stacked spatially varying deformable convolution block for joint image filtering. Our models can approximate several representative image smoothing filters with high accuracy comparable to state‐of‐the‐art methods, and serve as general tools for other joint image filtering tasks, such as color interpolation, depth map upsampling, saliency map upsampling, flash/non‐flash image denoising and RGB/NIR image denoising.