Learning confidence measures in the wild

Abstract

Confidence measures for stereo earned increasing popularity in most recent works concerning stereo, being effectively deployed to improve its accuracy. While most measures are obtained by processing cues from the cost volume, top-performing ones usually leverage on random-forests or CNNs to predict match reliability. Therefore, a proper amount of labeled data is required to effectively train such confidence measures. Being such ground-truth labels not always available in practical applications, in this paper we propose a methodology suited for training confidence measures in a self-supervised manner. Leveraging on a pool of properly selected conventional measures, we automatically detect a subset of very reliable pixels as well as a subset of erroneous samples from the output of a stereo algorithm. This strategy provides labels for training confidence measures based on machine-learning technique without ground-truth labels. Compared to state-of-the-art, our method is neither constrained to image sequences nor to image content. Experimental results on three challenging datasets with three stereo algorithms and three state-of-the-art confidence measures based on machine-learning techniques confirm the effectiveness of our proposal for self-supervised training.

Guided Stereo Matching

“…The focus of future work will be on devising strategies to guide our method without relying on active sensors. For instance, selecting reliable depth labels leveraging confidence measures [25] -since this strategy proved to be successful for self-supervised adaptation [36,37] and training learning-based confidence measures [38] -or from the output of a visual stereo odometry systems [41].…”

Motorcycles that See: Multifocal Stereo Vision Sensor for Advanced Safety Systems in Tilting Vehicles

“…Machine-learning approaches allow to use a set of metrics to improve the accuracy of the stereo confidences [ 86 , 87 , 88 , 89 ]. Recently, machine-learning approaches to big stereo data collected in adverse weather allowed for a self-supervised strategy that automatically labels confidence zones effectively [ 90 ].…”

Learning Monocular Depth Estimation Infusing Traditional Stereo Knowledge

“…Since for most tasks ground truth labels are difficult and expensive to source, some works recently enquired about the possibility to replace them with easier to obtain proxy labels. Tonioni et al [49] proposed to adapt deep stereo networks to unseen environments leveraging traditional stereo algorithms and confidence measures [43], Tosi et al [51] learned confidence estimation selecting positive and negative matches by means of traditional confidence measures, Makansi et al [33] and Liu et al [28] generated proxy labels for training optical flow networks using conventional methods. Specifically relevant to monocular depth estimation are the works proposed by Yang et al [58], using stereo visual odometry to train monocular depth estimation, by Klodt and Vedaldi [20], leveraging structure from motion algorithms and by Guo et al [13], obtaining labels from a deep network trained with supervision to infer disparity maps from stereo pairs.…”