Generalized Fringe-to-Phase Framework for Single-Shot 3D Reconstruction Integrating Structured Light with Deep Learning

Abstract

Three-dimensional (3D) shape acquisition of objects from a single-shot image has been highly demanded by numerous applications in many fields, such as medical imaging, robotic navigation, virtual reality, and product in-line inspection. This paper presents a robust 3D shape reconstruction approach integrating a structured-light technique with a deep learning-based artificial neural network. The proposed approach employs a single-input dual-output network capable of transforming a single structured-light image into two intermediate outputs of multiple phase-shifted fringe patterns and a coarse phase map, through which the unwrapped true phase distributions containing the depth information of the imaging target can be accurately determined for subsequent 3D reconstruction process. A conventional fringe projection technique is employed to prepare the ground-truth training labels, and part of its classic algorithm is adopted to preserve the accuracy of the 3D reconstruction. Numerous experiments have been conducted to assess the proposed technique, and its robustness makes it a promising and much-needed tool for scientific research and engineering applications.

Time-Distributed Framework for 3D Reconstruction Integrating Fringe Projection with Deep Learning

“…It should be noted that alternative output vectors, such as multiple phase-shifted fringe images or wrapped phases with different frequencies, can be used instead of numerators and denominators. However, recent studies [ 39 , 70 , 71 , 75 ] have demonstrated that the spatial F2ND approach yields similar results to the fringe-to-fringe approach while requiring less storage space due to fewer channels in the output vector. Moreover, the fringe-to-wrapped phase approach is not considered ideal as it produces inferior results compared with the spatial F2ND approach.…”

“…For instance, in the DFFS dataset, the first temporal output slice includes both numerators and denominators (i.e., [s,0,h,w,0] and [s,0,h,w,1]). In contrast, the second temporal output slice only consists of a single fringe order map [ 75 ] or a single coarse map [ 39 ] (i.e., [s,0,h,w,0]), resulting in a missing channel in the second temporal output slice.…”

“…The DFFS datasets consisted of a total of 2048 scenes with a resolution of [ 39 , 70 ]. Each scene involved the projection of 8 uniform sinusoidal four-step phase-shifted images, with two frequencies of and , by the projector.…”

“…Among various structured light techniques, fringe projection profilometry (FPP) is the most widely used technique in combination with the deep learning method for 3D reconstruction [ 38 , 39 , 40 , 41 , 42 ]. The integrated approaches can be broadly categorized into fringe-to-depth and fringe-to-phase techniques.…”

Three 1-bit speckle-embedded pulse-width modulation patterns for robust absolute 3D measurement

“…The wrapped phases of different frequency patterns are extracted by deep learning from a single composite fringe pattern to obtain the absolute phase. Nguyen et al [44] proposed a single-input dual-output network that obtains phase-shifted fringe images and coarse absolute phase maps from a single fringe pattern to achieve more accurate phase unwrapping. Compared to traditional methods, although the above methods based on deep learning accomplish 3D measurement with fewer patterns, they still use 8-bit sinusoidal patterns and lose the advantage of 1-bit patterns.…”