Multi range real-time depth inference from a monocular stabilized footage using a fully convolutional neural network

Pinard, Clément; Chevalley, Laure; Manzanera, Antoine; Filliat, David

doi:10.1109/ecmr.2017.8098659

Cited by 1 publication

(1 citation statement)

References 18 publications

(21 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, when running in inference, no pose estimation is needed, and only DepthNet is used. A similar algorithm as Pinard et al [18] can then be used to estimate absolute depth maps at a relatively low computational cost.…”

Section: Frame Stabilizationmentioning

confidence: 99%

Learning Structure-from-Motion from Motion

Pinard¹,

Chevalley²,

Manzanera³

et al. 2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

This work is based on a questioning of the quality metrics used by deep neural networks performing depth prediction from a single image, and then of the usability of recently published works on unsupervised learning of depth from videos. These works are all predicting depth from a single image, thus it is only known up to an undetermined scale factor, which is not sufficient for practical use cases that need an absolute depth map, i.e. the determination of the scaling factor. To overcome these limitations, we propose to learn in the same unsupervised manner a depth map inference system from monocular videos that takes a pair of images as input. This algorithm actually learns structure-frommotion from motion, and not only structure from context appearance. The scale factor issue is explicitly treated, and the absolute depth map can be estimated from camera displacement magnitude, which can be easily measured from cheap external sensors. Our solution is also much more robust with respect to domain variation and adaptation via fine tuning, because it does not rely entirely on depth from context. Two use cases are considered, unstabilized moving camera videos, and stabilized ones. This choice is motivated by the UAV (for Unmanned Aerial Vehicle) use case that generally provides reliable orientation measurement. We provide a set of experiments showing that, used in real conditions where only speed can be known, our network outperforms competitors for most depth quality measures. Results are given on the well known KITTI dataset [1], which provides robust stabilization for our second use case, but also contains moving scenes which are very typical of the in-car road context. We then present results on a synthetic dataset that we believe to be more representative of typical UAV scenes. Lastly, we present two domain adaptation use cases showing superior robustness of our method compared to single view depth algorithms, which indicates that it is better suited for highly variable visual contexts.

show abstract

Section: Frame Stabilizationmentioning

confidence: 99%