Untrained deep learning-based fringe projection profilometry

Abstract

Deep learning-based fringe projection profilometry (FPP) shows potential for challenging three-dimensional (3D) reconstruction of objects with dynamic motion, complex surface, and extreme environment. However, the previous deep learning-based methods are all supervised ones, which are difficult to be applied for scenes that are different from the training, thus requiring a large number of training datasets. In this paper, we propose a new geometric constraint-based phase unwrapping (GCPU) method that enables an untrained deep learning-based FPP for the first time. An untrained convolutional neural network is designed to achieve correct phase unwrapping through a network parameter space optimization. The loss function of the optimization is constructed by following the 3D, structural, and phase consistency. The designed untrained network directly outputs the desired fringe order with the inputted phase and fringe background. The experiments verify that the proposed GCPU method provides higher robustness compared with the traditional GCPU methods, thus resulting in accurate 3D reconstruction for objects with a complex surface. Unlike the commonly used temporal phase unwrapping, the proposed GCPU method does not require additional fringe patterns, which can also be used for the dynamic 3D measurement.

Hformer: Hybrid convolutional neural network transformer network for fringe order prediction in phase unwrapping of fringe projection

“…The datasets used in this work are from reference, 16 , 35 which includes fringe patterns captured by cameras and projector with a resolution of 640×480 pixels. Both single object and multiple objects are considered in the dataset.…”

“…In the experiment, we use two datasets to measure the performance namely the wrapped phase dataset A 16 and the wrapped phase dataset B 35 . The dataset A with a total number of samples of 1000 wrapped phase—fringe order pairs and 1000 fringe pattern—fringe order pairs.…”

“…Datasets were used for training, validation, and testing in this paper 16 , 35 . The source code is available at https://github.com/3D-measurement/Hformer.…”

“…In the experiment, we use two datasets to measure the performance namely the wrapped phase dataset A 16 and the wrapped phase dataset B. 35 The dataset A with a total number of samples of 1000 wrapped phase-fringe order pairs and 1000 fringe pattern-fringe order pairs. To validate the wrapped phase performance, we use 856 pairs of samples are used for training, 72 pairs of samples are used for validation, and 72 pairs of samples are used for the testing.…”

Deep image prior plus sparsity prior: toward single-shot full-Stokes spectropolarimetric imaging with a multiple-order retarder

“…Recently, state-of-the-art deep-image prior (DIP) methods using untrained networks have been proposed for image denoise and restoration with provable convergence 36 – 47 Therein, the structure of an untrained network with randomly initialized weights can function as a prior on image statistics without any training, mainly because deep neural networks are good at representing and generating realistic images 48 – 50 Specifically, an untrained network is paired with a physically differentiable forward-imaging model in which the network weights are updated through a loss function comparing the experimental measurement and the generated measurement from the network output passed through the forward imaging model.…”