Rectification Based Single-Shot Structured Light for Accurate and Dense 3D Reconstruction

Abstract


 Structured Light (SL) patterns generated based on pseudo-random arrays are widely used for single-shot 3D reconstruction using projector-camera systems. These SL images consist of a set of tags with different appearances, where these patterns will be projected on a target surface, then captured by a camera and decoded. The precision of localizing these tags from captured camera images affects the quality of the pixel-correspondences between the projector and the camera, and consequently that of the derived 3D shape. In this paper, we incorporate a quadrilateral representation for the detected SL tags that allows the construction of robust and accurate pixel-correspondences and the application of a spatial rectification module that leads to high tag classification accuracy. When applying the proposed method to single-shot 3D reconstruction, we show the effectiveness of this method over a baseline in estimating denser and more accurate 3D point-clouds.

P‐49: Student Poster: Efficient Direct‐Block‐Address Encoding for Single‐Shot‐Based 3D Reconstruction

“…We need to identify correspondences among some subset of the pixels in I and I . For this purpose the camera tag indices xĩ ,j , yĩ ,j are obtained using the detection and rectification method in [12]. The proposed scheme can obtain its corresponding projector tag indices in pixels, by directly estimating the block addressesĩ, and recover from them the pixel indices of its 3 × 3 tags as…”

“…Multi-shot SL methods [3][4][5][6] take advantage of multiple patterns that are projected sequentially over time, allowing simple encoding / decoding of projector pixel addresses. In contrast, single-shot SL is preferred in contexts of dynamically moving cameras, projectors or surfaces, and in scenarios where short projection time is important [7][8][9][10][11][12].…”

“…One of the most widely used single-shot methods are patterns based on tag embeddings, where tags having different colors, shapes and/or texture properties are used to represent an encoded SL pattern [7,8,[10][11][12]. In particular, many of these methods have utilized Pseudo-Random Arrays (PRA) for pattern encoding [7,[10][11][12], where the goal is to generate arrays such that every two-dimensional (2D) block (i.e., codeword) is unique, regardless of origin, however this uniqueness test of every newly added block is a computational bottleneck in PRA based SL image construction.…”

“…In order to obtain the correspondences between the projector and the camera pixels in the decoding stage, a search in the projector space [7,11,12] or in a look up table [10] needs to be performed, with a computational or memory requirement which grows as the display resolution increases. A different approach, but much less widely used, has been introduced for constructing SL patterns with tag embedding based on non-overlapping blocks [8], such that each of the non-overlapping blocks encodes its column address and, assuming that the relative positions of the projector and camera are known, the row addresses are obtained from the epipolar geometry.…”

“…Specifically, we partition an image into a number of nonoverlapping blocks and directly encoding both the row and column addresses, optionally with further error detection / correction capabilities, followed by a region growing approach in order to produce a denser set of projector-camera correspondences. In contrast to PRA based methods [7,11,12], our proposed approach does not require a uniqueness test during encoding, nor any sort of search or look-up during decoding. Thus, unlike existing PRA based methods [7,11,12], our proposed scheme offers attractive computational and memory efficiency, making it particularly suitable to high resolution projectors and/or dynamically moving / hand-held cameras.…”