Using Self-Contradiction to Learn Confidence Measures in Stereo Vision

Abstract

Learned confidence measures gain increasing importance for outlier removal and quality improvement in stereo vision. However, acquiring the necessary training data is typically a tedious and time consuming task that involves manual interaction, active sensing devices and/or synthetic scenes. To overcome this problem, we propose a new, flexible, and scalable way for generating training data that only requires a set of stereo images as input. The key idea of our approach is to use different view points for reasoning about contradictions and consistencies between multiple depth maps generated with the same stereo algorithm. This enables us to generate a huge amount of training data in a fully automated manner. Among other experiments, we demonstrate the potential of our approach by boosting the performance of three learned confidence measures on the KITTI2012 dataset by simply training them on a vast amount of automatically generated training data rather than a limited amount of laser ground truth data. * The research leading to these results has received funding from the EC FP7 project 3D-PITOTI (ICT-2011-600545) and from the Austrian Research Promotion Agency (FFG) as Vision+ project 836630 and together with OMICRON electronics GmbH as Bridge1 project 843450.

UAV-Based Autonomous Image Acquisition with Multi-view Stereo Quality Assurance by Confidence Prediction

“…vegetation) and a camera constellation, the MVS confidence encodes the likelihood that a dense reconstruction algorithm will work as intended. With "work as intended" we mean that if a scene part is observed by a sufficient number of cameras then the algorithm should be able to produce a 3D measurement within the theoretical uncertainty bounds for each pixel that observes this scene part [29]. The first matter we address in this section is how we can generate training data to predict the MVS confidence without any hard ground truth.…”

“…The first matter we address in this section is how we can generate training data to predict the MVS confidence without any hard ground truth. Therefore we extend our approach for stereo vision [29] to multiview stereo. Then we outline our machine learning setup and explain how we can use this setup to predict the MVS confidence in real-time during the image acquisition.…”

“…As it is extremely tedious to come by 3D ground truth, the basic idea of [29] is to use self-consistency and selfcontradiction from different view points for generating labeled training data. This approach is related to depthmap fusion, but outputs 2D label images instead of depthmaps.…”

“…In [29], we start by computing a depthmap for each stereo pair in the dataset. A single depthmap can be interpreted as the 3D reconstruction of a camera cluster with two cameras and a fixed baseline.…”

“…This MVS confidence prediction is related but not equal to the (two-view) stereo confidence prediction, which is a topic of increasing interest in the domain of stereo vision [15,46,33,29]. In stereo vision, the confidence encodes the likelihood that an already computed depth value is correct, whereas in our case it encodes the likelihood This paper was accepted to the 7th International Workshop on Computer Vision in Vehicle Technology (CVVT 2016) and will appear in IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2016.…”