Variational Fusion of Time-of-Flight and Stereo Data for Depth Estimation Using Edge-Selective Joint Filtering

Chen, Baoliang; Jung, Cheolkon

doi:10.1109/tmm.2018.2825883

Cited by 28 publications

(5 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Examples of this family are the methods of [32], that also uses confidence measures for the ToF and stereo vision systems to drive the process, and the works of Chen et Al. [33,34], that combines the variational approach with edge-preserving filtering.…”

Section: Related Workmentioning

confidence: 99%

Stereo and ToF data fusion by learning from synthetic data

et al. 2019

View full text Add to dashboard Cite

Time-of-Flight (ToF) sensors and stereo vision systems are both capable of acquiring depth information but they have complementary characteristics and issues. A more accurate representation of the scene geometry can be obtained by fusing the two depth sources. In this paper we present a novel framework for data fusion where the contribution of the two depth sources is controlled by confidence measures that are jointly estimated using a Convolutional Neural Network. The two depth sources are fused enforcing the local consistency of depth data, taking into account the estimated confidence information. The deep network is trained using a synthetic dataset and we show how the classifier is able to generalize to different data, obtaining reliable estimations not only on synthetic data but also on real world scenes. Experimental results show that the proposed approach increases the accuracy of the depth estimation on both synthetic and real data and that it is able to outperform state-of-the-art methods.

show abstract

Section: Related Workmentioning

confidence: 99%

Stereo and ToF data fusion by learning from synthetic data

et al. 2019

View full text Add to dashboard Cite

show abstract

“…D EPTH information is a core cue for context aware scene understanding, and is widely used in 3D modeling, augmented reality, and autonomous driving [1], [2], [3]. Depth images contain the distance of objects in the scene from the camera.…”

Section: Introductionmentioning

confidence: 99%

Single Image Depth Estimation Using Edge Extraction Network and Dark Channel Prior

Jung

Kim

2021

IEEE Access

Self Cite

View full text Add to dashboard Cite

The key to the depth estimation from a single image lies in inferring the distance of various objects without copying texture while maintaining clear object boundaries. In this paper, we propose depth estimation from a single image using edge extraction network and dark channel prior (DCP). We build an edge extraction network based on generative adversarial networks (GANs) to select valid depth edges from a number of edges in an image. We use DCP to generate a transmission map that is able to represent distance from the camera. Transmission map is generated by conducting minimum value filtering on DCP. First, we concatenate the transmission map with the original RGB image to form a tensor, i.e. RGB+T. Second, we generate an initial depth image from the tensor through the generator by inferring depth from stacked residual blocks. Third, we compare the edge map of the initial depth image with that of the input RGB image to select valid depth edges. Both edge maps are generated by the edge extraction network. Finally, we distinguish real and fake on the generated depth image using a discriminator, and enhances the performance of the generator. Various experiments on NYU, Make3D and MPI Sintel datasets demonstrate that the proposed network generates clear edges in depth images as well as outperforms state-of-the-art methods in terms of visual quality and quantitative measurements.INDEX TERMS Depth estimation, dark channel prior, deep learning, edge extraction, generative adversarial network, transmission.

show abstract

“…S TEREO matching is one of the most extensively studied topics in computer vision [1]- [3]. In general, stereo matching is composed of four steps: similarity cost calculation, cost aggregation, disparity selection, and disparity refinement [4], [5].…”

Section: Introductionmentioning

confidence: 99%

Quaternary Census Transform Based on the Human Visual System for Stereo Matching

Kim

Lim

et al. 2020

IEEE Access

View full text Add to dashboard Cite

The census transform is a non-parametric local transform that is widely used in stereo matching. This transform encodes the structural information of a local patch into a binary code stream representing the relative intensity ordering of the pixels within the patch. Despite its high performance in stereo matching, the census transform often generates identical binary code streams for two different patches because it simply thresholds the pixels within the patch at the center pixel intensity. To overcome this problem, we introduce a quaternary census transform that encodes the local structural information into a quaternary code stream by employing both the relative intensity ordering and the minimum visibility threshold of the human eye known as the just-noticeable difference. Moreover, because the human eye activates different areas of the retina based on brightness, the patch size for the proposed quaternary census transform adaptively varies depending on the luminance of each pixel. Experimental results on well-known Middlebury stereo datasets prove that the proposed transform outperforms the other census transform-based methods in terms of the accuracy of stereo matching. INDEX TERMS Census transform, depth estimation, disparity map, human visual system, similarity cost calculation, stereo image processing, stereo matching.

show abstract

Variational Fusion of Time-of-Flight and Stereo Data for Depth Estimation Using Edge-Selective Joint Filtering

Cited by 28 publications

References 21 publications

Stereo and ToF data fusion by learning from synthetic data

Stereo and ToF data fusion by learning from synthetic data

Single Image Depth Estimation Using Edge Extraction Network and Dark Channel Prior

Quaternary Census Transform Based on the Human Visual System for Stereo Matching

Contact Info

Product

Resources

About