Single-Image Depth Inference Using Generative Adversarial Networks

Abstract

Depth has been a valuable piece of information for perception tasks such as robot grasping, obstacle avoidance, and navigation, which are essential tasks for developing smart homes and smart cities. However, not all applications have the luxury of using depth sensors or multiple cameras to obtain depth information. In this paper, we tackle the problem of estimating the per-pixel depths from a single image. Inspired by the recent works on generative neural network models, we formulate the task of depth estimation as a generative task where we synthesize an image of the depth map from a single Red, Green, and Blue (RGB) input image. We propose a novel generative adversarial network that has an encoder-decoder type generator with residual transposed convolution blocks trained with an adversarial loss. Quantitative and qualitative experimental results demonstrate the effectiveness of our approach over several depth estimation works.

Towards Adversariality in Depth Estimation Based on Local Equilibrium Disparity Consensus and Ego-motion

“…We show that joint model optimization outperforms the use of only the (AS) model. Further, we compare our model with existing supervised depth estimation methods [36][28] [12], unsupervised depth estimation methods [33], Self-supervised depth estimation methods [15], and GAN depth estimation methods [14] As shown in tab 2. Compared to the state-of-the-art supervised method [37], our model learning process benefits from the adversarial training loss, which leads the network to improve itself over time in addition to the ego-motion that the (U) model provides.…”

“…We should note that we used two separate likewise (AS) models. We have seen better results than using only one as per [14] by training them adversarially. Also, the encoderdecoder used is different than of [14], as shown in [37], the encoder-decoder used produced state-of-the-art results; thus, we utilize it with our fine-tuned model.…”

“…The adversarial training process consists of training the models independently and sequentially. The Generators G (namely G 1 and G 2 ) of our models learn to estimate strong depth maps while the mutual discriminator D similarity becomes more robust, in contrast to the used model in [14] where only a single-Image is used. Our model's behavior creates a sort of adversariality between the two models that are further exploited to estimate the optimal depth map D f .…”

“…Specifically, we propose a framework consisting of a Supervised learning model (AS) that utilizes an encoder-decoder architecture [36] in it a Generative adversarial framework (from which our named model AS) by adding a discriminator that differentiates between the real and generated depth maps. Similar to [14], we instead train the model for both left-right image pairs that then go through an agreement process to choose the better depth map. Jointly, the unsupervised model uses the estimated depth map and the estimated global camera motion (ego-motion) between adjacent frames to reproduce a better disparity map.…”

Intelligent 3D Perception System for Semantic Description and Dynamic Interaction

“…Some are popular due to their low cost and easy data manipulation, such as color image (RGB image) provided by cameras. These sensors can be used to identify Augmented Reality Tags (AR-Tags) [11,12], among other computational vision tasks [13,14,15]. Despite its worldwide use, the RGB sensor provides a simple image of the environment, without any specific characteristic that allows smart actions.…”