OmniDepth: Dense Depth Estimation for Indoors Spherical Panoramas

Abstract

Recent work on depth estimation up to now has only focused on projective images ignoring 360 o content which is now increasingly and more easily produced. We show that monocular depth estimation models trained on traditional images produce sub-optimal results on omnidirectional images, showcasing the need for training directly on 360 o datasets, which however, are hard to acquire. In this work, we circumvent the challenges associated with acquiring high quality 360 o datasets with ground truth depth annotations, by re-using recently released large scale 3D datasets and re-purposing them to 360 o via rendering. This dataset, which is considerably larger than similar projective datasets, is publicly offered to the community to enable future research in this direction. We use this dataset to learn in an end-to-end fashion the task of depth estimation from 360 o images. We show promising results in our synthesized data as well as in unseen realistic images.

Spherical View Synthesis for Self-Supervised 360° Depth Estimation

“…Dataset: Given the unavailability of stereo 360 o datasets, we take a similar approach to [49] and render panoramas from displaced viewpoints in both vertical and horizontal placements as shown in Fig. 3.…”

“…The most apparent issue is the unavailability of data, and thus, two concurrent works addressed 360 o depth estimation by generating data via rendering existing 3D datasets. Two baseline models were presented in [49] after creating a large dataset of color and depth pairs using a mix of synthetic and real scenes. Further, [41] utilized the more recent advances in depth estimation from videos and rendered videos from a purely synthetic 3D dataset.…”

“…Qualitative results of each category of trained models (TC6 was chosen as it is the best performing trinocular model). From left to right: the input color image, the ground truth depth from [49], the fully supervised prediction (SV), the self-supervised predictions of horizontal (LR), vertical (UD) and trinocular (TC6) placements. Additional examples can be found in our supplementary material.…”

360° Surface Regression with a Hyper-Sphere Loss

“…To the best of our knowledge, there is no other similar work on monocular 360 o surface normal estimation. In an effort to show the importance of training directly on the omnidirectional domain, we provide comparisons of our model with learning-based methods trained on traditional perspec- Table 4: Quantitative results of our model trained on our dataset's train-split and evaluated on our test-split compared to the two neural network architectures for omnidirectional monocular depth estimation presented in [64] and the method of [61] re-trained on our dataset's train-set. We present the mean, median and root mean square angular error across our dataset's test-set.…”

“…They train their networks on cubemap projections of 360 o video sequences rendered from the SunCG [51] dataset. Moreover, in [64], the authors use an endto-end approach to learn 360 o depth from equirectangular indoors scenes. They present a dataset generated via rendering existing 3D datasets and two neural network architectures, one more typical and the other constructed with rectangular filters and dilated convolutions [59] to account for the distortion in the spherical domain.…”