High Quality Monocular Depth Estimation via Transfer Learning

Abstract

Accurate depth estimation from images is a fundamental task in many applications including scene understanding and reconstruction. Existing solutions for depth estimation often produce blurry approximations of low resolution. This paper presents a convolutional neural network for computing a high-resolution depth map given a single RGB image with the help of transfer learning. Following a standard encoder-decoder architecture, we leverage features extracted using high performing pre-trained networks when initializing our encoder along with augmentation and training strategies that lead to more accurate results. We show how, even for a very simple decoder, our method is able to achieve detailed high-resolution depth maps. Our network, with fewer parameters and training iterations, outperforms state-of-the-art on two datasets and also produces qualitatively better results that capture object boundaries more faithfully. Code and corresponding pre-trained weights are made publicly available 1 .

Towards Real-Time Monocular Depth Estimation for Robotics: A Survey

“…The best result of supervised learning-based method is shown in red and bold values, the best result of unsupervised learning-based method is shown in blue and bold values, the best result of semi-supervised learning-based method is shown in green and bold values, and the best result of domain adaptation method is shown in cyan and bold values. The results of Saxena et al [36] are reproduce from Eigen et al [20]; the running-time of Fu et al [89] is reported in Patil et al [158]; the running-time of Alhashim et al [86] is reported in Wang et al [113].…”

“…The stacked images are fed to an encoderdecoder architecture to learn depth from fused information. Based on DenseNet-169 [83], Alhashim and Wonka [86] design a densely connected encoder-decoder architecture. Unlike [22], they use a simple decoder method which consists of a bilinear upsampling and two convolution layers.…”

“…Experimental results on the NYU dataset show that the obtained depth estimation network runs at 178 fps on an Nvidia-TX2 GPU, while the RMSE and δ 1 values are 0.604 and 0.771 respectively. Inspired by the densely-connected encoder-decoder architecture [86], Wang et al [113] design a highly compact network named DepthNet Nano. DepthNet Nano applies densely connected projection batchnorm expansion projection (PBEP) modules to reduce network architecture and computation complexity while maintaining the representative ability.…”

GeoFill: Reference-Based Image Inpainting of Scenes with Complex Geometry

“…Predicting depth from a single image is an ill-posed problem. However, learning based approaches have shown impressive performance by either treating monocular depth estimation as a regression or classification task [1,13,15,16,18,19,22,23,44,45,58,59]. Recent advances include BTS [23], which introduces local planar guidance layers to guide the features to full resolution instead of standard upsampling layers during the decoding phase.…”

DenseTact: Optical Tactile Sensor for Dense Shape Reconstruction

“…Unlike general depth images from datasets such as NYU [31], the DenseTact dataset requires more focus on the local deformation information since the global information is similar between datasets. Therefore, a much simpler version of the network is selected with a pre-trained encoder and decoder with skip connections [32]. The encoder part of the network consists of a pre-trained DenseNet-161 [33].…”