Evaluation of Stereo Matching Costs on Images with Radiometric Differences

Abstract

Abstract-Stereo correspondence methods rely on matching costs for computing the similarity of image locations. We evaluate the insensitivity of different costs for passive binocular stereo methods with respect to radiometric variations of the input images. We consider both pixel-based and window-based variants like the absolute difference, the sampling-insensitive absolute difference, and normalized cross correlation, as well as their zero-mean versions. We also consider filters like LoG, mean, and bilateral background subtraction (BilSub) and non-parametric measures like Rank, SoftRank, Census, and Ordinal. Finally, hierarchical mutual information (HMI) is considered as pixelwise cost. Using stereo datasets with ground-truth disparities taken under controlled changes of exposure and lighting, we evaluate the costs with a local, a semi-global, and a global stereo method. We measure the performance of all costs in the presence of simulated and real radiometric differences, including exposure differences, vignetting, varying lighting and noise. Overall, the ranking of methods across all datasets and experiments appears to be consistent. Among the best costs are BilSub, which performs consistently very well for low radiometric differences; HMI, which is slightly better as pixel-wise matching cost in some cases and for strong image noise; and Census, which showed the best and most robust overall performance.

Depth Map Estimation Using Census Transform for Light Field Cameras

“…In depth estimation methods based on the census transform, a matching cost is often computed by using Hamming distance between B c of the center view and B s of the other views. However, this cost calculation method is affected by sensor noise as mentioned in [14]. To achieve the robustness against sensor noise, we compute a matching cost as follows:…”

“…To reduce the influence of the radiometric distortion caused by the vignetting effect of the micro-lenses, our initial depth estimation method computes matching costs in a window-based measure for sub-aperture images transformed by the census transform. Matching among two binarized images obtained by the census transform, however, has sensitivity against sensor noise as mentioned by Hirschmuler et al [14]. Therefore, we propose a cost calculation with majority operations to reduce the influence of sensor noise.…”

Synergy of stereo cloud top height and ORAC optimal estimation cloud retrieval: evaluation and application to AATSR

“…In terms of application to CTH retrieval, the most recent study is that by and involved the development of the M4 stereo image matching algorithm, which was influenced by the development of similar stereo algorithms applied to the MISR instrument . Here, based on work undertaken by Hirschmuller and Scharstein (2009), we apply the non-parametric census stereo algorithm (Zabih and Woodfill, 1994) to derive CTH. This approach has been demonstrated to be the most effective area-based stereo matcher for imagery with simulated radiometric distortions similar to those found in EO-derived data (Hirschmuller and Scharstein, 2009).…”

“…Here, based on work undertaken by Hirschmuller and Scharstein (2009), we apply the non-parametric census stereo algorithm (Zabih and Woodfill, 1994) to derive CTH. This approach has been demonstrated to be the most effective area-based stereo matcher for imagery with simulated radiometric distortions similar to those found in EO-derived data (Hirschmuller and Scharstein, 2009). As such, the census algorithm is applied in all cases in this study and is described in the following section.…”

Hierarchical Scan-Line Dynamic Programming for Optical Flow Using Semi-Global Matching

“…This function has been identified [9] to be very descriptive and robust, especially under strong illumination variations. Since this is a crucial feature for real-world applications the function is increasingly applied for both, stereo [16] and optical flow estimation methods [20].…”