ConvMatch: Rethinking Network Design for Two-View Correspondence Learning

Abstract

Multilayer perceptron (MLP) has been widely used in two-view correspondence learning for only unordered correspondences provided, and it extracts deep features from individual correspondence effectively. However, the problem of lacking context information limits its performance and hence, many extra complex blocks are designed to capture such information in the follow-up studies. In this paper, from a novel perspective, we design a correspondence learning network called ConvMatch that for the first time can leverage convolutional neural network (CNN) as the backbone to capture better context, thus avoiding the complex design of extra blocks. Specifically, with the observation that sparse motion vectors and dense motion field can be converted into each other with interpolating and sampling, we regularize the putative motion vectors by estimating dense motion field implicitly, then rectify the errors caused by outliers in local areas with CNN, and finally obtain correct motion vectors from the rectified motion field. Extensive experiments reveal that ConvMatch with a simple CNN backbone consistently outperforms state-of-the-arts including MLP-based methods for relative pose estimation and homography estimation, and shows promising generalization ability to different datasets and descriptors. Our code is publicly available at https://github.com/SuhZhang/ConvMatch.

ResMatch: Residual Attention Learning for Feature Matching

“…We evaluate our methods for two-view image matching on three datasets, including YFCC100M ( The performance is compared to NN with RT (Lowe 2004), learnable filter ConvMatch (Zhang and Ma 2023), and feature matching GNNs including SuperGlue (Sarlin et al 2020), SGMNet (Chen et al 2021), ParaFormer (Lu et al 2023a), and LightGlue (9 layers) (Lindenberger, Sarlin, and Pollefeys 2023). All GNNs are trained on GL3D.…”

“…PointCN (Yi et al 2018) takes an early effort to learn match filtering as a classification task. ConvMatch (Zhang and Ma 2023), an alternative, employs self-attention to model vector field consensus (Ma et al 2014), which shares a similar motivation with us. Instead of filtering putative sets, SuperGlue (Sarlin et al 2020) designs an attention-based GNN to match sparse features in a graph matching manner.…”