How Accurate Can Block Matches Be in Stereo Vision?

Abstract

Abstract. This article explores the sub-pixel accuracy attainable for the disparity computed from a rectified stereo pair of images with small baseline. In this framework we consider translations as the local deformation model between patches in the images. A mathematical study shows first how discrete block-matching can be performed with arbitrary precision under Shannon-Whittaker conditions. This study leads to the specification of a block-matching algorithm which is able to refine disparities with sub-pixel accuracy. Moreover, a formula for the variance of the disparity error caused by the noise is introduced and proved. Several simulated and real experiments show a decent agreement between this theoretical error variance and the observed RMSE in stereo pairs with good SNR and low baseline. A practical consequence is that under realistic sampling and noise conditions in optical imaging, the disparity map in stereo-rectified images can be computed for the majority of pixels (but only for those pixels with meaningful matches) with a 1/20 pixel precision.Key words. Block-matching, sub-pixel accuracy, noise error estimate.1. Introduction. Stereo algorithms aim at reconstructing a 3D model from two or more images of the same scene acquired from different angles. Assuming for a sake of simplicity that the cameras are calibrated, and that the image pair has been stereo-rectified, our work will focus on the matching process. The matching of stereo images has been studied in depth for decades. We refer to [43] and [6] for a complete comparison of different methods.Generally stereo matching methods are divided in two classes, the local algorithms and the global ones. Given two images of the same scene, the local methods compare a small block of pixels surrounding each pixel in the first image to the candidate blocks on the epipolar line in the second image. The blocks are usually compared by the normalized cross correlation (NCC) or the sum of squared differences (SSD). Having a minimum of the SSD does not guarantee at all that the match is correct. In general, only a significant proportion of the image can be reliably matched (about 40 to 80%). Block-matching methods can indeed produce wrong disparities near the intensity discontinuities in the images. This "fattening effect" is a classic problem in block-matching methods. It occurs when a salient image feature (typically an edge) lies within the comparison window ϕ but away from its center. This can produce a large error near points at which the disparity ε has a jump. Several papers have attempted with some success to alleviate this problem by using adaptive shape windows [21,48,22,18], adaptive support-weight windows [53], a barycentric correction [12], or by feature matching methods [44]. If the images of the stereo pair u 1 and u 2 have little aliasing, [12] showed that the recovered disparity map obtained by minimizing a continuous quadratic distance between u 1 and u 2 has therefore two main error terms: the fattening error, and the error due to noise. There ar...

Interactive Content Retargeting for Multi-Panel Displays

“…We used a similar idea of [32] to interpolate cost curve epi(d) to generate denser samples -ēpi(d). In order to find the minimum d' in the interpolated cost curve, we first calculate d'= argmin d ēpi(d), then getting mathematical minimum đ of the curve (the vertex of the curve) by fitting with a parabola y = ax² + bx + c using d' and its two closest points.…”

Performance evaluation in obstacle avoidance

“…Also in literature the performance of stereo vision is discussed for example by Sabater et al (2011), which evaluates the disparity error for several stereo algorithms. It compares stereo methods by calculating the Root-Mean Squared (RMS) error.…”

Analysis of Image Registration with Tangent Distance

“…We then generate a set of test patterns that lie between the transformation manifolds of the two reference patterns. The test patterns are generated such that their true class labels are given by the class label of the closer manifold as in (24). We then classify the test patterns with the tangent distance method by estimating the transformation parameters in one step using the low-pass filtered versions of the reference and test patterns.…”