Parametric scene model for geometry restoration using differential rendering method

Abstract

For the correct reconstruction of the real-world geometry, we propose an iterative approach that utilizes the method of differential rendering. This method is based on optimizing the parameters of scene geometry objects to approximate the synthesized scene images to their real images obtained, for example, through digital photography. For the optimization methods to work efficiently during rendering, a series of differential scene images corresponding to the increments of individual scene geometry parameters are formed. To construct the differential scene images, it is necessary to create a parametric model of its geometry. Since the scene geometry is usually represented as a triangular mesh, the triangle vertices become the parameters of the model. However, the number of triangles requires enormous computational resources, which may be impractical. We propose the parametric model of scene geometry, based on a combination of approximate voxel and refined triangular mesh representations. In this model, two levels of geometry modification are suggested. At the first level voxels are the directly modifiable parameters. At the second level individual or connected triangles can be modified precisely. The rendering process is divided into two parts. The first part involves calculating the luminance of direct illumination in the limited representation of the scene, solely related to the modified geometry or its shadow area. The second part involves calculating the luminance of indirect illumination. The calculation of differentials is based on the assumption that the luminance of indirect illumination remains almost unchanged under slight geometry modifications. These solutions significantly reduce the computation time of differentials while preserving the physical correctness of the calculation results and enabling the calculation of noise-free increments of differential scene images for a noisy base image.