Mind the Gap - A Benchmark for Dense Depth Prediction Beyond Lidar

Schilling, Hendrik; Gutsche, Marcel; Brock, Alexander; Spath, Dane; Rother, Carsten; Krispin, Karsten

doi:10.1109/cvprw50498.2020.00177

Cited by 4 publications

(1 citation statement)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The basic principle of depth enabled dataset construction is to use a device with reliable depth estimation capability, that won't be available during evaluation. Typically, we can use a rig with a RGB camera and a depth sensor like structured light [21], Time of Flight, embedded Lidar [34,35], or light-field camera grids [27]. For evaluation, only the camera will be available, and the evaluation step will then measure the agreement between "reliable depth" measured by the dedicated sensor and estimated depth.…”

Section: Constructing a Depth Enabled Datasetmentioning

confidence: 99%

Does it work outside this benchmark? Introducing the Rigid Depth Constructor tool, depth validation dataset construction in rigid scenes for the masses

Pinard¹,

Manzanera²

2021

Preprint

View full text Add to dashboard Cite

We present a protocol to construct your own depth validation dataset for navigation. This protocol, called RDC for Rigid Depth Constructor, aims at being more accessible and cheaper than already existing techniques, requiring only a camera and a Lidar sensor to get started. We also develop a test suite to get insightful information from the evaluated algorithm. Finally, we take the example of UAV videos, on which we test two depth algorithms that were initially tested on KITTI [34] and show that the drone context is dramatically different from in-car videos. This shows that a single context benchmark should not be considered reliable, and when developing a depth estimation algorithm, one should benchmark it on a dataset that best fits one's particular needs, which often means creating a brand new one. Along with this paper we provide the tool with an open source implementation and plan to make it as user-friendly as possible, to make depth dataset creation possible even for small teams. Our key contributions are the following: We propose a complete, open-source and almost fully automatic software application for creating validation datasets with densely annotated depth, adaptable to a wide variety of image, video and range data. It includes selection tools to adapt the dataset to specific validation needs, and conversion tools to other dataset formats. Using this application, we propose two new real datasets, outdoor and indoor, readily usable in UAV navigation context. Finally as examples, we show an evaluation of two depth prediction algorithms, using a collection of comprehensive (e.g. distribution based) metrics.

show abstract

Section: Constructing a Depth Enabled Datasetmentioning

confidence: 99%