Real-time stereo vision system using adaptive weight cost aggregation approach

Abstract

Many vision applications require high-accuracy dense disparity maps in real time. Due to the complexity of the matching process, most real-time stereo applications rely on local algorithms in the disparity computation. These local algorithms generally suffer from matching ambiguities as it is difficult to find appropriate support for each pixel. Recent research shows that algorithms using adaptive cost aggregation approach greatly improve the quality of disparity map. Unfortunately, although these improvements are excellent, they are obtained at the expense of high computational. This article presents a hardware implementation for speeding up these methods. With hardware friendly approximation, we demonstrate the feasibility of implementing this expensive computational task on hardware to achieve real-time performance. The entire stereo vision system, includes rectification, stereo matching, and disparity refinement, is realized using a single field programmable gate array. The highly parallelized pipeline structure makes system be capable to achieve 51 frames per second for 640 × 480 stereo images. Finally, the success of accuracy improvement is demonstrated on the Middlebury dataset, as well as tests on real scene.

High-Speed Low-Complexity Guided Image Filtering-Based Disparity Estimation

“…However the concern with the local matching algorithms is its low precision. To mitigate this issue, investigations have been made to implement dedicated hardware architectures of more precise algorithms, such as Semi Global Matching (SGM) [8], [9] and Adaptive Support Weight (ADSW) [10], [11]. For the past few years, hardware implementations predicated on SGM and ADSW algorithms have become the preferred solution towards higher matching precision in embedded vision applications [5], [7], [12], [13].…”

“…The VLSI design of an ADSW algorithm that adopted the mini-Census transform was implemented to improve the accuracy and robustness of the system to radiometric distortions [14]. Incorporating an ADSW algorithm and integration of pre and post-processing units, [11] proposed the implementation of a complete stereo vision system. Finally, a hardware oriented stereo matching system based on the adaptive Census transform is presented in [10].…”

Runtime Performance and Power Optimization of Parallel Disparity Estimation on Many-Core Platforms

“…However, these algorithms are prone to disparity errors at depth discontinuity regions due to the use of a fixed local window shape and size [Yoon and Kweon 2006]. To improve matching accuracy, a few attempts have been made by combining or modifying existing algorithms and transforms [Ambrosch and Kubinger 2010;Baha and Larabi 2012;Zhang et al 2009], the most recent Adaptive Support Weight (ADSW) methods are currently the most accurate [Gehrig et al 2009;Ding et al 2011;Perri et al 2013]. They work by assigning different weights to the pixels in the support window based on their color or proximity to the central pixel.…”

“…However, they typically have complex implementations with high memory and hardware demands which have the potential to limit scalability to higher resolution images. As a result, investigations have been made to implement dedicated hardware architectures of more precise algorithms, such as Semi Global Matching (SGM) [Gehrig et al 2009;Banz et al 2010] and Adaptive Support Weight (ADSW) [Ding et al 2011;Perri et al 2013].…”

A Fuzzy Logic Approach for Stereo Matching Suited for Real-Time Processing

“…In [18], a two step method was proposed where an initial disparity was computed and only reliable disparity values were aggregated to the cost function for the second step. Jin et al [19] proposed a cost aggregation method that computed ground control points (GCP) in order to refine disparity values using the GCPs, while [20] adapted the support area using assumptions of similarity and proximity values. Using the graphics hardware, real-time processing can be achieved, but the implementation must be efficiently adapted and modifications to that structure will be difficult to carry out.…”