GMS: Grid-Based Motion Statistics for Fast, Ultra-robust Feature Correspondence

Abstract

Feature matching aims at generating correspondences across images, which is widely used in many computer vision tasks. Although considerable progress has been made on feature descriptors and fast matching for initial correspondence hypotheses, selecting good ones from them is still challenging and critical to the overall performance. More importantly, existing methods often take a long computational time, limiting their use in real-time applications. This paper attempts to separate true correspondences from false ones at high speed. We term the proposed method (GMS) grid-based motion Statistics, which incorporates the smoothness constraint into a statistic framework for separation and uses a grid-based implementation for fast calculation. GMS is robust to various challenging image changes, involving in viewpoint, scale, and rotation. It is also fast, e.g., take only 1 or 2 ms in a single CPU thread, even when 50K correspondences are processed. This has important implications for real-time applications. What's more, we show that incorporating GMS into the classic feature matching and epipolar geometry estimation pipeline can significantly boost the overall performance. Finally, we integrate GMS into the well-known ORB-SLAM system for monocular initialization, resulting in a significant improvement. Keywords Feature matching • Epipolar geometry • Visual SLAM • Structure-from-motion • GMS 1 Introduction Feature matching is one of the most fundamental problems in the computer vision community. It aims to generate correspondences across different views of an object or scene, which is widely used in many vision tasks such as structurefrom-motion (Schonberger and Frahm 2016) and Visual SLAM (Davison et al. 2007; Mur-Artal et al. 2015). Typical solutions rely on feature detectors (Harris and stephens 1988), descriptors (Lowe 2004), and matchers (Muja and Communicated by Jiri Matas.

Neural-Guided RANSAC: Learning Where to Sample Model Hypotheses

“…They use their approach with SIFT as well as LIFT [55] features. We include results for DeMoN [50], a learned SfM pipeline, and GMS [2], a semi-dense approach using ORB features [38]. As classical baselines, we compare to vanilla RANSAC [12] and USAC [34].…”

NM-Net: Mining Reliable Neighbors for Robust Feature Correspondences

“…For example, a locality preserving matching algorithm is presented in [23], which assumes that local geometric structures in the vicinity of inliers are invariant under rigid/non-rigid transformations, where the spatially knn search is utilized to represent variations of local structures. The spatially local information is exploited in a statistical manner in [3]. The similarity of local regions between two images is measured by the number of correspondences; all correspondences located in the regions are considered inliers if the number is larger than a predefined threshold.…”

“…step to ensure correct matches and improve the accuracy [3]. This paper focuses on a learning-based approach toward selecting correct matches from an initial set of feature correspondences [36].…”

“…Spatial positions of matched features are discrete and irregular (note that RGB or texture information is not available any more). To effectively mine the consistency from raw positions, spatially local information is often employed in previous hand-crafted algorithms [20,3,23]. Indeed, spatially local information has played an important role in image segmentation [21] and point cloud segmentation [26].…”

Localizing Discriminative Visual Landmarks for Place Recognition

“…Here, we use the center of each region as its coordinate. Generally, correct matches should have similar coordinate differences and incorrect matches have random coordinate differences [19], [20]. Therefore, a twodimensional histogram is built by analyzing the coordinate differences in both x and y direction of all landmark pairs.…”