Review of Wide-Baseline Stereo Image Matching Based on Deep Learning

Abstract

Strong geometric and radiometric distortions often exist in optical wide-baseline stereo images, and some local regions can include surface discontinuities and occlusions. Digital photogrammetry and computer vision researchers have focused on automatic matching for such images. Deep convolutional neural networks, which can express high-level features and their correlation, have received increasing attention for the task of wide-baseline image matching, and learning-based methods have the potential to surpass methods based on handcrafted features. Therefore, we focus on the dynamic study of wide-baseline image matching and review the main approaches of learning-based feature detection, description, and end-to-end image matching. Moreover, we summarize the current representative research using stepwise inspection and dissection. We present the results of comprehensive experiments on actual wide-baseline stereo images, which we use to contrast and discuss the advantages and disadvantages of several state-of-the-art deep-learning algorithms. Finally, we conclude with a description of the state-of-the-art methods and forecast developing trends with unresolved challenges, providing a guide for future work.

Binocular Images Dense Matching considering Image Adaptive Color Weights and Feature Points

“…e last two terms in (10) are smoothing terms. e energy function is used to calculate the whole image from a global perspective [35].…”

“…(2) e proposed CBT method is used to obtain the initial matching cost volume; (3) SIFT feature matching is performed on binocular images to acquire feature points; (4) e feature points are optimized to obtain feature point coordinates information and accurate disparity information; (5) According to the information of a priori feature points, the cost volume is optimized. Except for the accurate disparity of the feature points, the corresponding cost values of the other candidate disparities are set to be invalid; (6) Cost aggregation of path constraints is performed; (7) WTA method is used to obtain the disparity map based on the left image; (8) Left and right consistency detection is carried out; (9) e final disparity map is output; (10) e quality of the disparity map is evaluated. 5 and 6) in Middlebury public data set were selected for experiments in the paper.…”

“…Finally, similarity matching is performed on the extracted features. At present, there are many commonly used feature point matching algorithms, such as Harris [7], Forster [8], SIFT (Scale-invariant Feature Transform) [9], and deep learning methods [10]. Due to the features of rotation invariance, scale invariance and a ne invariance [11], SIFT has become a commonly used algorithm in feature matching.…”

Quasi-Dense Matching Algorithm for Close-Range Image Combined With Feature Line Constraint

“…Image matching aims to establish the correspondence between corresponding points on images acquired from different viewpoints of the same object scene and is the first crucial step in acquiring three-dimensional (3D) information on the basis of images and then realizing 3D reconstruction [1]. This method is widely applied in computer vision and other fields, such as 3D vision, robotics navigation, image stitching, gesture recognition, and video tracking.…”

The Fusion Strategy of 2D and 3D Information Based on Deep Learning: A Review

“…For example, the image includes abundant 2D texture information but fails to represent the geometric information. Even though the 3D information can be acquired by post-processing the data collected by some mature visual-based perceptions, such as the mono- [13] and stereo camera [14], the quality of the This paper aims to introduce up-to-date and representative fusion strategies for 2D and 3D information fusion based on deep learning. In Section 2, we briefly introduce the basic terminologies and background of the deep-learning techniques for classification, detection, and segmentation.…”