Learned Multi-View Texture Super-Resolution

Abstract

We present a super-resolution method capable of creating a high-resolution texture map for a virtual 3D object from a set of lower-resolution images of that object. Our architecture unifies the concepts of (i) multi-view superresolution based on the redundancy of overlapping views and (ii) single-view super-resolution based on a learned prior of high-resolution (HR) image structure. The principle of multi-view super-resolution is to invert the image formation process and recover the latent HR texture from multiple lower-resolution projections. We map that inverse problem into a block of suitably designed neural network layers, and combine it with a standard encoder-decoder network for learned single-image super-resolution. Wiring the image formation model into the network avoids having to learn perspective mapping from textures to images, and elegantly handles a varying number of input views. Experiments demonstrate that the combination of multi-view observations and learned prior yields improved texture maps.

Geometry-Aware Reference Synthesis for Multi-View Image Super-Resolution

“…Several traditional approaches [7,38] and learning-based approaches [20,28] study the multi-view texture SR problem. However, they only super-resolve the texture of the object instead of the entire image.…”

“…Existing multi-view SR related methods rarely consider this problem. Some methods focus on multi-view texture [28] or light field images [50] and some other methods [19,24] require high-resolution multiview as a reference, which is not easily available in practice due to high storage costs and bandwidth constraints. The most similar method to ours is SASRnet [37].…”

3D Reconstruction and Texture Refinement with Image Enhancement for Improved Visual Quality

“…In (W. , (Kim et al, 2019), (Fu et al, 2018), and (Rouhani et al, 2018), texture optimization techniques have been adopted to improve the quality of the final texture, reducing ghosting and blurring problems. More recently, Deep Learning techniques have been adopted (Huang et al, 2020), (Richard et al, 2020), (Y. , and (Wu et al, 2019), with which results outperform the quality of previous methods.…”

Super-Resolution Appearance Transfer for 4D Human Performances

“…Li et al [35] modified the architecture of EDSR [37] to exploit the information of both texture maps and normal maps of objects. Richard et al [46] combined a redundancy-based part with a prior-based part in a network to create new texture maps. The first method requires the creation of normal maps, a process that introduces heavy computational cost for a high number of frames.…”