A Wavelet-Based Multiresolution Approach to Solve the Stereo Correspondence Problem Using Mutual Information

Abstract

In this correspondence, we propose a wavelet-based hierarchical approach using mutual information (MI) to solve the correspondence problem in stereo vision. The correspondence problem involves identifying corresponding pixels between images of a given stereo pair. This results in a disparity map, which is required to extract depth information of the relevant scene. Until recently, mostly correlation-based methods have been used to solve the correspondence problem. However, the performance of correlation-based methods degrades significantly when there is a change in illumination between the two images of the stereo pair. Recent studies indicate MI to be a more robust stereo matching metric for images affected by such radiometric distortions. In this short correspondence paper, we compare the performances of MI and correlation-based metrics for different types of illumination changes between stereo images. MI, as a statistical metric, is computationally more expensive. We propose a wavelet-based hierarchical technique to counter the increase in computational cost and show its effectiveness in stereo matching.

Evaluation of Stereo Matching Costs on Images with Radiometric Differences

“…MI has been first used for stereo matching by Chrastek and Jan [18], but with disappointing results. Later work on MI in window-based stereo methods [19]- [21] demonstrated its power to model complex radiometric relationships. Others used Publication in IEEE Transactions on Pattern Analysis and Machine Intelligence c 2009 IEEE approximations of MI [22] for a segment-wise stereo matching.…”

“…Others used Publication in IEEE Transactions on Pattern Analysis and Machine Intelligence c 2009 IEEE approximations of MI [22] for a segment-wise stereo matching. It has been found [20], [21] that large windows are needed for collecting enough data for the required joint probability distribution, but large windows again result in blurring at object boundaries. Therefore, Fookes et al [20] proposed a hierarchical method for estimating probability priors over the whole image at a lower resolution.…”

“…They concluded that ZNCC and Rank performs best on images without radiometric changes, while the performance of MI is best on images with artificially changed radiometry. Sarkar and Bansal [21] compared MI and SSD for window-based matching on images with ground truth and artificial radiometric changes. They found that MI handles radiometric differences well, but its performance depends heavily on the window size.…”

“…In contrast to previous studies [21], [36]- [38], we test all costs not only for window-based matching, but (where applicable) also for pixelbased matching with a semi-global method (SGM) and graph cuts (GC) as a strong global method. Furthermore, in addition to simulated global and local radiometric changes, we perform experiments on stereo pairs with real radiometric differences.…”

“…However, typical windows of 9×9 or 11×11 pixel do not contain enough pixels for deriving meaningful probability distributions [20], [21]. Larger windows would be needed, but larger windows are known to increase blurring of discontinuities [10].…”