Simultaneous Estimation of Object Region and Depth in Participating Media Using a ToF Camera

Abstract

Three-dimensional (3D) reconstruction and scene depth estimation from 2-dimensional (2D) images are major tasks in computer vision. However, using conventional 3D reconstruction techniques gets challenging in participating media such as murky water, fog, or smoke. We have developed a method that uses a continuous-wave time-of-flight (ToF) camera to estimate an object region and depth in participating media simultaneously. The scattered light observed by the camera is saturated, so it does not depend on the scene depth. In addition, received signals bouncing off distant points are negligible due to light attenuation, and thus the observation of such a point contains only a scattering component. These phenomena enable us to estimate the scattering component in an object region from a background that only contains the scattering component. The problem is formulated as robust estimation where the object region is regarded as outliers, and it enables the simultaneous estimation of an object region and depth on the basis of an iteratively reweighted least squares (IRLS) optimization scheme. We demonstrate the effectiveness of the proposed method using captured images from a ToF camera in real foggy scenes and evaluate the applicability with synthesized data.

Time-of-Flight Imaging in Fog Using Polarization Phasor Imaging

“…If ϕ s θ || , x = ϕ s (θ ⊥ , x), PP s is a constant and same as the degree of polarization. Combining the above equations, the scattering phasor can be calculated from arg ⋅ operator is used to calculated the angle of the pha region, the recovered depth is messy due to the weak reflected suppressed using an image mask [21,22,27]. The image mask ca segmentation, and we implemented the maximum between-class It can be observed from Equation (7) that PP s is a critical factor to estimate the scattering component.…”

“…where arg(•) operator is used to calculated the angle of the phasor. In the background region, the recovered depth is messy due to the weak reflected light, which is usually suppressed using an image mask [21,22,27]. The image mask can be obtained by image segmentation, and we implemented the maximum between-class variance method [42] on the recovered amplitude image to mask the background of the depth image.…”

“…In this work, we only consider the MPI caused by scattering media. To correct MPI generated by scattering media, many methods have been studied and developed, including convolutional sparse coding [18], data-driven prior [19], temporal properties [20], iterative optimization algorithms [21,22], image filtering [23], polarization properties of light [24,25], and phasor imaging [26,27]. Kijima D. et al used multiple time-gating to estimate the scattering property of fog with a short-pulse ToF camera, for reconstructing the depth and intensity of a scene [20], which is not applicable to CW-ToF cameras.…”