Weakly-supervised contrastive learning in path manifold for Monte Carlo image reconstruction

Abstract

Image-space auxiliary features such as surface normal have significantly contributed to the recent success of Monte Carlo (MC) reconstruction networks. However, path-space features, another essential piece of light propagation, have not yet been sufficiently explored. Due to the curse of dimensionality, information flow between a regression loss and high-dimensional path-space features is sparse, leading to difficult training and inefficient usage of path-space features in a typical reconstruction framework. This paper introduces a contrastive manifold learning framework to utilize path-space features effectively. The proposed framework employs weakly-supervised learning that converts reference pixel colors to dense pseudo labels for light paths. A convolutional path-embedding network then induces a low-dimensional manifold of paths by iteratively clustering intra-class embeddings, while discriminating inter-class embeddings using gradient descent. The proposed framework facilitates path-space exploration of reconstruction networks by extracting low-dimensional yet meaningful embeddings within the features. We apply our framework to the recent image- and sample-space models and demonstrate considerable improvements, especially on the sample space. The source code is available at https://github.com/Mephisto405/WCMC.

Morphological Anti-Aliasing Method for Boundary Slope Prediction

“…Aliasing is a common problem in both offline rendering [28,27,12,11,10,7,8,14,13,9] and real-time rendering [17,19,2,21,30,18]. Traditional anti-aliasing algorithms, supersampling anti-aliasing and multi-sampling anti-aliasing algorithms, have been the standard for antialiasing algorithms for more than ten years.…”

Extension: Adaptive Sampling with Implicit Radiance Field

“…The training dataset consists of 20 different scenes with randomly generated viewpoints and randomly modified scene lighting, materials and textures, resulting in a total of about 300 scenes. For each configuration, Ground Truth light field data with different sampling numbers and resolutions are generated, and the number of sampling points in each block is (1,2,4,8,16) five kinds, including The resolution is (16 1 , 16 2 , 16 3 , 16 4 , 16 5 ) five kinds. These data were first used to train the R network to converge and well reconstruct sparse samples into dense, noise-free light fields.…”

“…By training a neural network denoiser through offline learning, it can filter noisy Monte Carlo rendering results into high-quality smooth output, greatly improving physics-based Availability of rendering techniques [13], common research includes predicting a filtering kernel based on g-buffer [2], using GAN to generate more realistic filtering results [28], and analyzing path space features Perform manifold contrastive learning to enhance the rendering effect of reflections [4], use weight sharing to quickly predict the rendering kernel to speed up reconstruction [6], filter and reconstruct high-dimensional incident radiation fields for unbiased reconstruction Drawing guide [12], etc. 2.…”

A Virtual Point Light Generation Method in Close-Range Area

“…By training a neural network denoiser through offline learning, it can filter noisy Monte Carlo rendering re-sults into high-quality smooth output, greatly improving physics-based Availability of rendering techniques [13], common research includes predicting a filtering kernel based on g-buffer [2], using GAN to generate more realistic filtering results [29], and analyzing path space features Perform manifold contrastive learning to enhance the rendering effect of reflections [3], use weight sharing to quickly predict the rendering kernel to speed up reconstruction [7], filter and reconstruct high-dimensional incident radiation fields for unbiased reconstruction rendering guide [12], etc. 2.…”