Correspondence matching among stereo images with object flow and minimum spanning tree aggregation

Abstract

Stereoscopic correspondence matching is applied in many applications like robot navigation, automatic driving, virtual, and augmented reality by reconstructing the scene in three-dimensional environments. In the most real scenes, the moving objects attract more attentions than static objects and background. Thus, temporal information of consecutive frames like motion flow has been proven to improve the matching accuracy as weight prior. In this article, we propose a costaggregation method joining object flow and minimum spanning tree-based support region rather than aggregating on fixed size windows. However, directly combining object flow and minimum spanning tree filtering is not straightforward, due to the extremely high computing complexity. The proposed scheme implements nonlocal cost aggregation with object-based optical flow, which extends the idea of the minimum spanning tree and flow-based motion estimation to increase the matching accuracy. Temporal evidence of object flow is not only used in minimum spanning tree support region building but also incorporated with one hybrid edge prior to optimize the disparity estimation. The experimental results with synthetic stereo videos show that the proposed method outperforms other state-of-the-art stereo matching methods in most data sets. The whole stereo correspondence algorithm achieves competitive performance in terms of both accuracy and speed. We also illustrate the performance of the proposed method with the Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) benchmark as one extensive comparison.

mproved Stereo Matching Algorithm based on Census Transform and Dynamic Histogram Cost Computation

“…A combination of minimum spanning tree based on support region and joining object flow in cost aggregation is adopted by [24]- [27] rather than using fixed-sized windows. An enhanced MST called 3DMST-CM was proposed by [28] handling a cases based on ambiguity of image pixel to achieve a high level accuracy in disparity map.…”

“…Post-processing in the stereo matching taxonomy is called disparity refinement to remove any outliers, uncertainties, and noise from the map for disparity to achieve a greater level of accuracy. These noises only can be detected by performing a left-right consistency checking, bi-modality, match goodness jumps, and occlusion constraint [27]. Additionally, [32] proposed the technique using cross voting of image-based, and a median filter is applied to perfect the depth estimation cost using the triple image approach to identify textureless regions and false matches.…”

3-D Vision for Robot Perception

“…Zhang et al propose a cost-aggregation method joining object flow and minimum spanning tree-based support region rather than aggregating on fixed size windows. 10 The scheme implements nonlocal cost aggregation with object-based optical flow, which extends the idea of the minimum spanning tree and flow-based motion estimation to increase the matching accuracy. Temporal evidence of object flow is not only used in minimum spanning tree support region building but also incorporated with one hybrid edge prior to optimize the disparity estimation.…”

Evaluation of Stereo Images Matching

“…With the advancement of technology, image matching techniques have become increasingly important in a variety of applications, including military affairs [7], medicine [8], industry [9], license plate recognition [10], fingerprint recognition [11,12], face recognition [13], animal motion trajectory tracking system [14], and face tracking shooting system [15]. Image matching represents a principal aspect of many problems in computer vision, including motion tracking [16], object recognition and matching [17,18], 3D reconstruction [19], stereo correspondence [20], image classification and retrieval [21], and camera calibration [22].…”