Depth prediction from a single image with conditional adversarial networks

Jung, Hyungjoo; Kim, Young-Jung; Min, Dongbo; Oh, Chang-Hwan

doi:10.1109/icip.2017.8296575

Cited by 36 publications

(33 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…to [6]. Compared to [18], it does not require the use of a Monte Carlo method to capture the uncertainty of the model and improve performance, like [17].…”

Section: Methodsmentioning

confidence: 99%

“…Also, we bring a new insight to the use of the adversarial loss which requires a large amount of data to be effective. The network, which eventually get close to best performances on NYUv2, is also much simpler to train with respect to [14,6] as it can be performed end-to-end.…”

Section: Related Workmentioning

confidence: 96%

“…They work by defining an adversarial loss which is modeled by a network that classifies the likeliness of the output, often regularized with a L 1 term. Jung et al [6] successfully used this idea of the adversarial loss to perform depth prediction with a two-phase training strategy: network is first trained with a L 1 and posteriorly fine-tuned with the adversarial loss.…”

Section: Related Workmentioning

confidence: 99%

“…The encoder and decoder parts are connected through skip connections like proposed by [21] to improve context-aware learning. In contrast to precedent architectures [14,6], our network can be trained in a single phase and does not require any additional analytical model like CRFs [4,14].…”

Section: Deep Depth Prediction Networkmentioning

confidence: 99%

“…The first main challenge faced by the aforementioned papers is defining an appropriate loss function for depth regression. L 2 has often been a popular choice for this task, but a custom loss [1] and, more recently, an adversarial loss [6] have also been adopted with success. The second challenge concerns the network architecture, which usually followed the advances proposed every year in this flourishing field: VGG16 [2,3], fully convolutional encoderdecoders [7], and recently Residual Networks (ResNet) [8].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

On Regression Losses for Deep Depth Estimation

Carvalho¹,

Saux²,

Trouvé-Peloux³

et al. 2018

2018 25th IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

Depth estimation from a single monocular image has reached great performances thanks to recent works based on deep networks. However, as various choices of losses, architectures and experimental conditions are proposed in the literature, it is difficult to establish their respective influence on the performances. In this paper we propose an in-depth study of various losses and experimental conditions for depth regression, on NYUv2 dataset. From this study we propose a new network for depth estimation combining an encoder-decoder architecture with an adversarial loss. This network reaches top ones state of the art on NUYv2 dataset while being simpler to train in a single phase.

show abstract

“…to [6]. Compared to [18], it does not require the use of a Monte Carlo method to capture the uncertainty of the model and improve performance, like [17].…”

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 96%

Section: Related Workmentioning

confidence: 99%

Section: Deep Depth Prediction Networkmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

On Regression Losses for Deep Depth Estimation

Carvalho¹,

Saux²,

Trouvé-Peloux³

et al. 2018

2018 25th IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

show abstract

Deep Depth from Defocus: How Can Defocus Blur Improve 3D Estimation Using Dense Neural Networks?

Carvalho

Saux

Trouvé-Peloux

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Depth estimation is of critical interest for scene understanding and accurate 3D reconstruction. Most recent approaches in depth estimation with deep learning exploit geometrical structures of standard sharp images to predict corresponding depth maps. However, cameras can also produce images with defocus blur depending on the depth of the objects and camera settings. Hence, these features may represent an important hint for learning to predict depth. In this paper, we propose a full system for single-image depth prediction in the wild using depth-fromdefocus and neural networks. We carry out thorough experiments to test deep convolutional networks on real and simulated defocused images using a realistic model of blur variation with respect to depth. We also investigate the influence of blur on depth prediction observing model uncertainty with a Bayesian neural network approach. From these studies, we show that out-of-focus blur greatly improves the depth-prediction network performances. Furthermore, we transfer the ability learned on a synthetic, indoor dataset to real, indoor and outdoor images. For this purpose, we present a new dataset containing real all-focus and defocused images from a Digital Single-Lens Reflex (DSLR) camera, paired with ground truth depth maps obtained with an active 3D sensor for indoor scenes. The proposed approach is successfully validated on both this new dataset and standard ones as NYUv2 or Depth-in-the-Wild. Code and new datasets are available at https:// github.com/ marcelampc/ d3net depth estimation.

show abstract

Unsupervised single-shot depth estimation using perceptual reconstruction

Angermann

Schwab

Haltmeier

et al. 2023

Machine Vision and Applications

View full text Add to dashboard Cite

Real-time estimation of actual object depth is an essential module for various autonomous system tasks such as 3D reconstruction, scene understanding and condition assessment. During the last decade of machine learning, extensive deployment of deep learning methods to computer vision tasks has yielded approaches that succeed in achieving realistic depth synthesis out of a simple RGB modality. Most of these models are based on paired RGB-depth data and/or the availability of video sequences and stereo images. However, the lack of RGB-depth pairs, video sequences, or stereo images makes depth estimation a challenging task that needs to be explored in more detail. This study builds on recent advances in the field of generative neural networks in order to establish fully unsupervised single-shot depth estimation. Two generators for RGB-to-depth and depth-to-RGB transfer are implemented and simultaneously optimized using the Wasserstein-1 distance, a novel perceptual reconstruction term, and hand-crafted image filters. We comprehensively evaluate the models using a custom-generated industrial surface depth data set as well as the Texas 3D Face Recognition Database, the CelebAMask-HQ database of human portraits and the SURREAL dataset that records body depth. For each evaluation dataset, the proposed method shows a significant increase in depth accuracy compared to state-of-the-art single-image transfer methods.

show abstract

Depth prediction from a single image with conditional adversarial networks

Cited by 36 publications

References 14 publications

On Regression Losses for Deep Depth Estimation

On Regression Losses for Deep Depth Estimation

Deep Depth from Defocus: How Can Defocus Blur Improve 3D Estimation Using Dense Neural Networks?

Unsupervised single-shot depth estimation using perceptual reconstruction

Contact Info

Product

Resources

About