Leveraging Confident Points for Accurate Depth Refinement on Embedded Systems

Abstract

Despite the notable progress in stereo disparity estimation, algorithms are still prone to errors in challenging conditions. Thus, heuristic disparity refinement techniques are usually deployed to improve accuracy. Moreover, stateof-the-art methods rely on complex CNNs requiring power hungry GPUs not suited for many practical applications constrained by limited computing resources. In this paper, we propose a novel technique for disparity refinement leveraging on confidence measures and a novel, automatic learning-based selection method to discard outliers. Then, a non-local strategy infers missing disparities by analyzing the closest reliable points. This framework is very fast and does not require any hand-tuned thresholding. We assess the performance of our Non-Local Anchoring (NLA), standalone refinement techniques and methods leveraging on confidence measures inside the stereo algorithm. Our evaluation with two popular stereo algorithms shows that our proposal significantly ameliorates their accuracy on Middlebury v3 and KITTI 2015 datasets. Moreover, since our method relies only on cues computed in the disparity domain, it is suited even for COTS stereo cameras coupled with embedded systems, e.g. nVidia Jetson TX2.

FC-DCNN: A densely connected neural network for stereo estimation

“…F. Tosi et al [21] use a confidence map in order to detect reliable and unreliable points in a disparity map. After removing the unreliable points from the map, they then update these unreliable points by aggregating the first reliable points along different paths that they call "anchor".…”

Joint Graph-based Depth Refinement and Normal Estimation

“…Local methods are characterized by a greedy approach. Tosi et al [33] adopt a two step strategy. First, the input disparity map is used to compute a binary confidence mask that classifies each pixel as reliable or not.…”

“…In particular, for each non reliable pixel, a set of anchor pixels with a reliable disparity is selected and the pixel disparity is estimated as a weighted average of the anchor disparities. Besides its low computational requirements, the method in [33] suffers two major drawbacks. On the one hand, pixels classified as reliable are left unchanged: this does not permit to correct possible pixels misclassified as reliable, which may bias the refinement of those pixels.…”

“…On the one hand, pixels classified as reliable are left unchanged: this does not permit to correct possible pixels misclassified as reliable, which may bias the refinement of those pixels. On the other hand, the method in [33], and local methods in general, cannot take advantage of the reliable parts of the disparity map fully, due to their local perspective.…”

“…In general, the refinement step is guided by the image associated to the measured or estimated depth map. The depth refinement literature mostly focuses on enforcing some kind of first order smoothness among the depth map pixels, possibly avoiding smoothing across the edges of the guide image that may correspond to object boundaries [1], [33], [35]. Although depth maps are typically piece-wise smooth, Mattia Rossi, Mireille EL Gheche and Pascal Frossard are with Signal Processing Laboratory (LTS4), Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland.…”