Progressive Correspondence Pruning by Consensus Learning

Zhao, Chen; Ge, Yixiao; Zhu, Feng; Zhao, Rui; Li, Hongsheng; Salzmann, Mathieu

doi:10.1109/iccv48922.2021.00640

Cited by 36 publications

(44 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Second, the architecture first uses DNNS for feature matching. Then, the architecture uses CLNet 49 to remove false matches to get the final matches. The first highlight is that an object‐aware block is proposed to compute a weighted feature map, which can guide the feature extraction.…”

Section: The Proposed Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Object‐aware deep feature extraction for feature matching

Li,

Wang,

Lai

et al. 2023

Concurrency and Computation

View full text Add to dashboard Cite

SummaryFeature extraction is a fundamental step in the feature matching task. A lot of studies are devoted to feature extraction. Recent researches propose to extract features by pre‐trained neural networks, and the output is used for feature matching. However, the quality and the quantity of the features extracted by these methods are difficult to meet the requirements for the practical applications. In this article, we propose a two‐stage object‐aware‐based feature matching method. Specifically, the proposed object‐aware block predicts a weighted feature map through a mask predictor and a prefeature extractor, so that the subsequent feature extractor pays more attention to the key regions by using the weighted feature map. In addition, we introduce a state‐of‐the‐art model estimation algorithm to align image pair as the input of the object‐aware block. Furthermore, our method also employs an advanced outlier removal algorithm to further improve matching quality. Experimental results show that our object‐aware‐based feature matching method improves the performance of feature matching compared with several state‐of‐the‐art methods.

show abstract

Section: The Proposed Methodsmentioning

confidence: 99%

“…Therefore, it is important to remove incorrect matches. In order to remove mismatches, the recently proposed CLNet 49 is employed in our proposed architecture. Figure 5B is the result obtained by the corresponding pruning block.…”

Section: Correspondence Pruning Blockmentioning

confidence: 99%

Object‐aware deep feature extraction for feature matching

Li,

Wang,

Lai

et al. 2023

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…We compare ConvMatch with both classical outlier rejection methods such as CRC (Fan et al 2021), GMS (Bian et al 2017), LPM (Ma et al 2019), VFC (Ma et al 2014), and learning-based methods such as PointCN (Yi et al 2018), OANet (Zhang et al 2019), CLNet (Zhao et al 2021), LMCNet (Liu et al 2021), MS 2 DGNet (Dai et al 2022). For outdoor relative pose estimation, we compare our method with the feature matching method SuperGlue (Sarlin et al 2020) additionally.…”

Section: Relative Pose Estimationmentioning

confidence: 99%

“…The Thirty-Seventh AAAI Conference on Artificial Intelligence techniques. After PointCN (Yi et al 2018) first considering the outlier rejection as a binary classification problem under the multilayer perceptron (MLP) framework, many other algorithms are proposed with the similar MLP-based network, e.g., PointACN (Sun et al 2020), OANet (Zhang et al 2019), LMCNet (Liu et al 2021), and CLNet (Zhao et al 2021). The essential reason for such a unified network design is the sparse and unordered nature of point correspondences, and only MLP can extract deep features stably in this case.…”

Section: Introductionmentioning

confidence: 99%

ConvMatch: Rethinking Network Design for Two-View Correspondence Learning

Zhang

2023

AAAI

View full text Add to dashboard Cite

Multilayer perceptron (MLP) has been widely used in two-view correspondence learning for only unordered correspondences provided, and it extracts deep features from individual correspondence effectively. However, the problem of lacking context information limits its performance and hence, many extra complex blocks are designed to capture such information in the follow-up studies. In this paper, from a novel perspective, we design a correspondence learning network called ConvMatch that for the first time can leverage convolutional neural network (CNN) as the backbone to capture better context, thus avoiding the complex design of extra blocks. Specifically, with the observation that sparse motion vectors and dense motion field can be converted into each other with interpolating and sampling, we regularize the putative motion vectors by estimating dense motion field implicitly, then rectify the errors caused by outliers in local areas with CNN, and finally obtain correct motion vectors from the rectified motion field. Extensive experiments reveal that ConvMatch with a simple CNN backbone consistently outperforms state-of-the-arts including MLP-based methods for relative pose estimation and homography estimation, and shows promising generalization ability to different datasets and descriptors. Our code is publicly available at https://github.com/SuhZhang/ConvMatch.

show abstract

“…OANet [9] maps matches to a set of clusters in a soft assignment manner, for local context exploring. CLNet [11] adopts a local-to-global consensus learning strategy to filter outliers progressively. LMCNet [12] and ConvMatch [15] learn the coherence and smoothness of the motion filed for outlier rejection.…”

Section: Outlier Rejectionmentioning

confidence: 99%

HetGNN-SF: Self-supervised learning on heterogeneous graph neural network via semantic strength and feature similarity

Liu

Yan

et al. 2023

Appl Intell

View full text Add to dashboard Cite

Accurately matching local features between a pair of images corresponding to the same 3D scene is a challenging computer vision task. Previous studies typically utilize attentionbased graph neural networks (GNNs) with fully-connected graphs over keypoints within/across images for visual and geometric information reasoning. However, in the context of feature matching, a significant number of keypoints are non-repeatable due to occlusion and failure of the detector, and thus irrelevant for message passing. The connectivity with non-repeatable keypoints not only introduces redundancy, resulting in limited efficiency (quadratic computational complexity w.r.t. the keypoint number), but also interferes with the representation aggregation process, leading to limited accuracy. Targeting towards high accuracy and efficiency, we propose MaKeGNN, a sparse attention-based GNN architecture which bypasses non-repeatable keypoints and leverages matchable ones to guide compact and meaningful message passing. More specifically, our Bilateral Context-Aware Sampling (BCAS) Module first dynamically samples two small sets of well-distributed keypoints with high matchability scores from the image pair. Then, our Matchable Keypoint-Assisted Context Aggregation (MKACA) Module regards sampled informative keypoints as message bottlenecks and thus constrains each keypoint only to retrieve favorable contextual information from intra-and inter-matchable keypoints, evading the interference of irrelevant and redundant connectivity with non-repeatable ones. Furthermore, considering the potential noise in initial keypoints and sampled matchable ones, the MKACA module adopts a matchability-guided attentional aggregation operation for purer data-dependent context propagation. By these means, we achieve the state-of-the-art performance on relative camera estimation, fundamental matrix estimation, and visual localization, while significantly reducing computational and memory complexity compared to typical attentional GNNs.

show abstract

Progressive Correspondence Pruning by Consensus Learning

Cited by 36 publications

References 38 publications

Object‐aware deep feature extraction for feature matching

Object‐aware deep feature extraction for feature matching

ConvMatch: Rethinking Network Design for Two-View Correspondence Learning

HetGNN-SF: Self-supervised learning on heterogeneous graph neural network via semantic strength and feature similarity

Contact Info

Product

Resources

About