Polarized Object Surface Reconstruction Algorithm Based on RU-GAN Network

Abstract

There are six possible solutions for the surface normal vectors obtained from polarization information during 3D reconstruction. To resolve the ambiguity of surface normal vectors, scholars have introduced additional information, such as shading information. However, this makes the 3D reconstruction task too burdensome. Therefore, in order to make the 3D reconstruction more generally applicable, this paper proposes a complete framework to reconstruct the surface of an object using only polarized images. To solve the ambiguity problem of surface normal vectors, a jump-compensated U-shaped generative adversarial network (RU-Gan) based on jump compensation is designed for fusing six surface normal vectors. Among them, jump compensation is proposed in the encoder and decoder parts, and the content loss function is reconstructed, among other approaches. For the problem that the reflective region of the original image will cause the estimated normal vector to deviate from the true normal vector, a specular reflection model is proposed to optimize the dataset, thus reducing the reflective region. Experiments show that the estimated normal vector obtained in this paper improves the accuracy by about 20° compared with the previous conventional work, and improves the accuracy by about 1.5° compared with the recent neural network model, which means the neural network model proposed in this paper is more suitable for the normal vector estimation task. Furthermore, the object surface reconstruction framework proposed in this paper has the characteristics of simple implementation conditions and high accuracy of reconstructed texture.

Passive Polarized Vision for Autonomous Vehicles: A Review

“…To manage ambiguity between diffuse or specular dominating reflection [ 317 ] they were the first to introduce deep learning and to provide a lighting invariance algorithm based on shape from shading. Yang et al [ 318 ] succeeded in using deep learning to reconstruct an object with shape from polarization information only. Knowing the polarization pattern of the blue sky [ 319 ] can also help to determine the object’s shape but this method is not suitable in real time.…”