Learning Monocular Depth Estimation Infusing Traditional Stereo Knowledge

Abstract

Depth estimation from a single image represents a fascinating, yet challenging problem with countless applications. Recent works proved that this task could be learned without direct supervision from ground truth labels leveraging image synthesis on sequences or stereo pairs. Focusing on this second case, in this paper we leverage stereo matching in order to improve monocular depth estimation. To this aim we propose monoResMatch, a novel deep architecture designed to infer depth from a single input image by synthesizing features from a different point of view, horizontally aligned with the input image, performing stereo matching between the two cues. In contrast to previous works sharing this rationale, our network is the first trained end-to-end from scratch. Moreover, we show how obtaining proxy ground truth annotation through traditional stereo algorithms, such as Semi-Global Matching, enables more accurate monocular depth estimation still countering the need for expensive depth labels by keeping a self-supervised approach. Exhaustive experimental results prove how the synergy between i) the proposed monoResMatch architecture and ii) proxy-supervision attains state-of-theart for self-supervised monocular depth estimation. The code is publicly available at https

Self-Supervised Monocular Depth Hints

“…Similar to before, we also show results for our method without pretraining ("Ours HR Resnet50 w/o pretraining"). Our non-pretrained model beats SuperDepth [26] in six out of seven metrics (tied in one), and compares favourably to the concurrent work monoResMatch [30], which makes use of a significantly more complex network compared to our encoder-decoder architecture.…”

“…The concurrent work monoResMatch by Tosi et al [30] also exploits proxy ground truth labels generated with a traditional stereo matching method [13]. The inclusion of the proxy supervision is shown to greatly improve accuracy over using a standard self-supervised loss.…”

“…Similarly, Zhu et al [43] add a supervised loss [1] to solve for optical flow and Kuznietsov et al [18] add a supervised loss for depth estimation from LiDAR. Concurrently proposed monoResMatch [30] uses this method to incorporate a proxy-supervised signal, albeit using a reverse Huber loss [19] as opposed to log L 1 . The addition of supervised losses change the objective function that is being minimized; one could view the additional term as a form of regularization, constraining the network prediction to adhere to the proposed depth values.…”

Deep Learning for Real-Time 3D Multi-Object Detection, Localisation, and Tracking: Application to Smart Mobility

“…Among methods tested which are dedicated to monocular cameras, we can find approaches such as SfmLearner [28], MonoResMatch [29], Monodepth [14] and Monodepth2 [30]. Except [28], which is trained on monocular image sequences by learning the structure from motion elements using three frames snippet, all of these algorithms require a pair of images from a calibrated stereoscopic camera in order to be able to do the training.…”

Enhancing Self-Supervised Monocular Depth Estimation with Traditional Visual Odometry

“…On the other hand, while existing works leverage such knowledge at training time only, we deploy a monocular VO algorithm to obtain geometrical priors to feed our network with. Being such priors sourced by a monocular setup, they are available at inference time in contrast to others available from stereo images [40,46] and thus available at training time only.…”