Depth error compensation for camera fusion system

Abstract

When the three-dimensional (3-D) video system includes a multiview video generation technique using depth data to provide more realistic 3-D viewing experiences, accurate depth map acquisition is an important task. In order to generate the precise depth map in real time, we can build a camera fusion system with multiple color cameras and one time-of-flight (TOF) camera; however, this method is associated with depth errors, such as depth flickering, empty holes in the warped depth map, and mixed pixels around object boundaries. In this paper, we propose three different methods for depth error reduction to minimize such depth errors. In order to reduce depth flickering in the temporal domain, we propose a temporal enhancement method using a modified joint bilateral filtering at the TOF camera side. Then, we fill the empty holes in the warped depth map by selecting a virtual depth and applying a weighted depth filtering method. After hole filling, we remove mixed pixels and replace them with new depth values using an adaptive joint multilateral filter. Experimental results show that the proposed method reduces depth errors significantly in near real time. © The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

Three-dimensional recovering based on variable focal length via Bezier networks

“…Therefore, these systems take a long time to achieve measurements of variable range. [1][2][3][4][5][6][7][8][9][10][11][12][13][14] Moreover, the recalibration increases 0?3% initial measurement error. 16 Additionally, the re-calibration of camera position and focal length is not achieved when the references do not exist during vision task.…”

“…The surface depth is provided by electromechanical device, which moves the vision system from h 0 to h 1 , h 2 , …, h n . From these surfaces, the line positions x 00 , x 10 , x 20 , …, x n0 are determined by detecting the intensity maximum via equation (11) and (12). Thus, the image centre is computed via equation (1) base on the data h 1 , h 2 , x 00 , x 10 and x 20 .…”

“…This procedure improves the measurements of variable range, which adjust the vision parameters via external re-calibration. [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17] The measurements of variable range via variable focal length are described in the section on 'Measurements of variable range via variable focal length'.…”

“…[8][9][10] Moreover, the rigid configuration produces occlusions and low resolution. 11,12 Recently, lighting methods such as fringe pattern, line and spot have been implemented by means of a flexible set-up to provide measurements of variable range. 1,13,14 For instance, the line projection moves the camera and the focal length to perform measurements of variable range.…”

“…11,12,19 These limitations reduce the capability of the traditional measurements of variable range. [9][10][11][12][13][14][15][16][17][18][19] The proposed method performs measurements of variable range via variable focal length without recalibration during the vision task. This flexible measurement method is carried out via Bezier networks based on the line position and the focal length.…”