Accurate Depth Extraction Method for Multiple Light-Coding-Based Depth Cameras

Pan, Yu; Liu, Rongke; Guan, Boshen; Du, Qiuchen; Xiong, Zixiang

doi:10.1109/tmm.2016.2646179

Cited by 16 publications

(10 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many depth optimization algorithms have been proposed in the literature, including the interpolation method [44], Markov random field (MRF) propagation [8], locally linear embedding (LLE) [2] and sub-pixel-based matching (SPM) [45]. However, these algorithms refine the depth map based on its intrinsic characteristics, which is not suitable for viewpoint images with narrow baselines.…”

Section: Sai-guided Depth Optimizationmentioning

confidence: 99%

Light-field compression using a pair of steps and depth estimation

An¹,

Cao²,

Liu³

et al. 2019

Opt. Express

View full text Add to dashboard Cite

Advanced handheld plenoptic cameras are being rapidly developed to capture information about light fields (LFs) from the 3D world. Rich LF data can be used to develop dense sub-aperture images (SAIs) that can provide a more immersive experience for users. Unlike conventional 2D images, 4D SAIs contain both the positional and directional information of light rays; the practical applications of handheld plenoptic cameras are limited by the huge volume of data required to capture this information. Therefore, an efficient LF compression method is vital for further application of the cameras. To this end, the pair of steps and depth estimation (PoS&DE) method is proposed in this paper, and the multiview video and depth (MVD) coding structure is used to relieve the LF coding burden. More specifically, a precise depth-estimation approach is presented for SAIs based on the cost function, and an SAI-guided depth optimization algorithm is designed to refine the initial depth map based on pixel variation tendency. Meanwhile, to reduce running time, intermediate SAI synthesis quality and coding bitrates, including the key SAIs selected and cost-computation steps, are set via extensive statistical experiments. In this way, only a limited number of optimally selected SAIs and their corresponding depth maps must be encoded. The experimental results demonstrate that our proposed LF compression solution using PoS&DE can obtain a satisfied coding performance.

show abstract

Section: Sai-guided Depth Optimizationmentioning

confidence: 99%

Light-field compression using a pair of steps and depth estimation

An¹,

Cao²,

Liu³

et al. 2019

Opt. Express

View full text Add to dashboard Cite

show abstract

“…Moreover, depth sensors illuminate a scene by infrared light, which could be unacceptable in many applications. The abovementioned problems limit the possible applications of depth sensors in FTV and VN systems, although depth cameras and lidars have recently undergone many improvements [58], [59]. Thus, the considerations of this paper are focused on depth estimation by multiview video analysis.…”

Section: Introductionmentioning

confidence: 99%

Depth Map Estimation for Free-Viewpoint Television and Virtual Navigation

2020

View full text Add to dashboard Cite

The paper presents a new method of depth estimation, dedicated for free-viewpoint television (FTV) and virtual navigation (VN). In this method, multiple arbitrarily positioned input views are simultaneously used to produce depth maps characterized by high interview and temporal consistencies. The estimation is performed for segments and their size is used to control the trade-off between the quality of depth maps and the processing time of depth estimation. Additionally, an original technique is proposed for the improvement of temporal consistency of depth maps. This technique uses the temporal prediction of depth, thus depth is estimated for P-type depth frames. For such depth frames, temporal consistency is high, whereas estimation complexity is relatively low. Similarly, as for video coding, I-type depth frames with no temporal depth prediction are used in order to achieve robustness. Moreover, we propose a novel parallelization technique that significantly reduces the estimation time. The method is implemented in C++ software that is provided together with this paper, so other researchers may use it as a new reference for their future works. In performed experiments, MPEG methodology was used whenever possible. The provided results demonstrate the advantages over the Depth Estimation Reference Software (DERS) developed by MPEG. The fidelity of a depth map, measured by the quality of synthesized views, is higher on average by 2.6 dB. This significant quality improvement is obtained despite a significant reduction of the estimation time, on average 4.5 times. The application of the proposed temporal consistency enhancement method increases this reduction to 29 times. Moreover, the proposed parallelization results in the reduction of the estimation time up to 130 times (using 6 threads). As there is no commonly accepted measure of the consistency of depth maps, the application of compression efficiency of depth is proposed as a measure of depth consistency.

show abstract

“…After all, humans rely heavily on the disparity of the images formed by our two eyes to perceive depth. Consequently, most prior works utilized stereo based techniques and reduced the problem into finding point correspondences and disparity matching [17,18,19,20]. However, requiring two cameras can be limiting, which is why researchers became creative and came up with various techniques to vary scene and shooting conditions in order to obtain two or more slightly different images of the same scene.…”

Section: Introductionmentioning

confidence: 99%

Single-Image Depth Inference Using Generative Adversarial Networks

Tan

Yao

Ruiz

et al. 2019

Sensors

View full text Add to dashboard Cite

Depth has been a valuable piece of information for perception tasks such as robot grasping, obstacle avoidance, and navigation, which are essential tasks for developing smart homes and smart cities. However, not all applications have the luxury of using depth sensors or multiple cameras to obtain depth information. In this paper, we tackle the problem of estimating the per-pixel depths from a single image. Inspired by the recent works on generative neural network models, we formulate the task of depth estimation as a generative task where we synthesize an image of the depth map from a single Red, Green, and Blue (RGB) input image. We propose a novel generative adversarial network that has an encoder-decoder type generator with residual transposed convolution blocks trained with an adversarial loss. Quantitative and qualitative experimental results demonstrate the effectiveness of our approach over several depth estimation works.

show abstract

Accurate Depth Extraction Method for Multiple Light-Coding-Based Depth Cameras

Cited by 16 publications

References 22 publications

Light-field compression using a pair of steps and depth estimation

Light-field compression using a pair of steps and depth estimation

Depth Map Estimation for Free-Viewpoint Television and Virtual Navigation

Single-Image Depth Inference Using Generative Adversarial Networks

Contact Info

Product

Resources

About