Dense Monocular Depth Estimation in Complex Dynamic Scenes

Ranftl, René; Vineet, Vibhav; Chen, Qifeng; Koltun, Vladlen

doi:10.1109/cvpr.2016.440

Cited by 170 publications

(139 citation statements)

References 29 publications

Supporting

Mentioning

138

Contrasting

Unclassified

Order By: Relevance

“…As a second category, single-view depth estimation ('LRC [19]' or 'DORN [14]') and optical flow estimation ('MirrorFlow [30] and 'HD 3 -F [72]' used parts of the dataset for training, these methods are disregarded for ranking. The third group comprises the multi-body or non-rigid SfM-based methods DMDE [51] and S.Soup [36]. The fourth category consists of the methods MFA [37], Mono-Stixel [5] and our Mono-SF approach, which are methods that fuse single-view depth information with multi-view geometry.…”

Section: Monocular Scene Flowmentioning

confidence: 99%

Mono-SF: Multi-View Geometry Meets Single-View Depth for Monocular Scene Flow Estimation of Dynamic Traffic Scenes

Brickwedde

Abraham

Mester

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Existing 3D scene flow estimation methods provide the 3D geometry and 3D motion of a scene and gain a lot of interest, for example in the context of autonomous driving. These methods are traditionally based on a temporal series of stereo images. In this paper, we propose a novel monocular 3D scene flow estimation method, called Mono-SF. Mono-SF jointly estimates the 3D structure and motion of the scene by combining multi-view geometry and singleview depth information. Mono-SF considers that the scene flow should be consistent in terms of warping the reference image in the consecutive image based on the principles of multi-view geometry. For integrating single-view depth in a statistical manner, a convolutional neural network, called ProbDepthNet, is proposed. ProbDepthNet estimates pixelwise depth distributions from a single image rather than single depth values. Additionally, as part of ProbDepth-Net, a novel recalibration technique for regression problems is proposed to ensure well-calibrated distributions. Our experiments show that Mono-SF outperforms state-of-theart monocular baselines and ablation studies support the Mono-SF approach and ProbDepthNet design.

show abstract

Section: Monocular Scene Flowmentioning

confidence: 99%

Mono-SF: Multi-View Geometry Meets Single-View Depth for Monocular Scene Flow Estimation of Dynamic Traffic Scenes

Brickwedde

Abraham

Mester

2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

show abstract

“…3) We solve the scale-indeterminacy issue, which is an inherent ambiguity for 3D reconstruction under perspective projection, while Taylor et al [38] method does not suffer from this, at the cost of being restricted to the orthographic camera model. Recently, Russel et al [39] and Ranftl et al [1] used objectlevel segmentation for dense dynamic 3D reconstruction. In contrast, our method is free from object segmentation, hence circumvent the difficulty associated with motion segmentation in a dynamic setting.…”

Section: )mentioning

confidence: 99%

“…While the conceptual idea of our work appeared in ICCV 2017, this journal version provides (i) in-depth realization of our overall optimization (ii) Qualitative comparison with [1], Video-PopUp [39] as well as statistical comparison with deep-learning method [43]. (v) Detail discussion on the failure cases, choice of euclidean metric for nearest neighbor graph construction, and limitation of our work with possible direction for improvements.…”

Section: )mentioning

confidence: 99%

“…SF, MF, TF refers to single frame, multi-frame and two-frame based approach respectively. The reference to the method DT [58], GLRT [57], BMM [12], PTA [56], DMDE [1]. Fig.…”

Section: Performance Analysismentioning

confidence: 99%

“…

…”

mentioning

confidence: 99%

See 2 more Smart Citations

Superpixel Soup: Monocular Dense 3D Reconstruction of a Complex Dynamic Scene

Kumar

Dai

2021

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

This work addresses the task of dense 3D reconstruction of a complex dynamic scene from images. The prevailing idea to solve this task is composed of a sequence of steps and is dependent on the success of several pipelines in its execution [1]. To overcome such limitations with the existing algorithm, we propose a unified approach to solve this problem. We assume that a dynamic scene can be approximated by numerous piecewise planar surfaces, where each planar surface enjoys its own rigid motion, and the global change in the scene between two frames is as-rigid-as-possible (ARAP). Consequently, our model of a dynamic scene reduces to a soup of planar structures and rigid motion of these local planar structures. Using planar over-segmentation of the scene, we reduce this task to solving a "3D jigsaw puzzle" problem. Hence, the task boils down to correctly assemble each rigid piece to construct a 3D shape that complies with the geometry of the scene under the ARAP assumption. Further, we show that our approach provides an effective solution to the inherent scale-ambiguity in structure-from-motion under perspective projection. We provide extensive experimental results and evaluation on several benchmark datasets. Quantitative comparison with competing approaches shows state-of-the-art performance.Index Terms-Dense 3D reconstruction, perspective camera, as-rigid-as-possible, relative scale ambiguity, structure from motion. !

show abstract

OmniDepth: Dense Depth Estimation for Indoors Spherical Panoramas

Zioulis

Karakottas

Zarpalas

et al. 2018

Computer Vision – ECCV 2018

157

246

View full text Add to dashboard Cite

Recent work on depth estimation up to now has only focused on projective images ignoring 360 o content which is now increasingly and more easily produced. We show that monocular depth estimation models trained on traditional images produce sub-optimal results on omnidirectional images, showcasing the need for training directly on 360 o datasets, which however, are hard to acquire. In this work, we circumvent the challenges associated with acquiring high quality 360 o datasets with ground truth depth annotations, by re-using recently released large scale 3D datasets and re-purposing them to 360 o via rendering. This dataset, which is considerably larger than similar projective datasets, is publicly offered to the community to enable future research in this direction. We use this dataset to learn in an end-to-end fashion the task of depth estimation from 360 o images. We show promising results in our synthesized data as well as in unseen realistic images.

show abstract

Dense Monocular Depth Estimation in Complex Dynamic Scenes

Cited by 170 publications

References 29 publications

Mono-SF: Multi-View Geometry Meets Single-View Depth for Monocular Scene Flow Estimation of Dynamic Traffic Scenes

Mono-SF: Multi-View Geometry Meets Single-View Depth for Monocular Scene Flow Estimation of Dynamic Traffic Scenes

Superpixel Soup: Monocular Dense 3D Reconstruction of a Complex Dynamic Scene

OmniDepth: Dense Depth Estimation for Indoors Spherical Panoramas

Contact Info

Product

Resources

About