Repeatability Is Not Enough: Learning Affine Regions via Discriminability

Mishkin, Dmytro; Radenović, Filip; Matas, Jiřı́

doi:10.1007/978-3-030-01240-3_18

Cited by 187 publications

(186 citation statements)

References 48 publications

Supporting

Mentioning

175

Contrasting

Order By: Relevance

“…keypoint detection). AffNet [27] applied a similar strategy to learn affinecovariant regions. Polar Transformer Networks [12] were used by [11] to build scale-invariant descriptors by transforming the input patch into log-polar space.…”

Section: Related Workmentioning

confidence: 99%

Linearized Multi-Sampling for Differentiable Image Transformation

Jiang

Sun

Tagliasacchi

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Figure 1: Image alignment example -We iteratively find the image transformation parameters that produce the downsampled target image when applied to the source image using different sampling strategies. We visualize the transformed source image at convergence when using bilinear sampling and the proposed linearized sampling. We further display the optimization path -iterations -for each method. The proposed linearized sampling provides improved gradients, leading to better convergence. AbstractWe propose a novel image sampling method for differentiable image transformation in deep neural networks. The sampling schemes currently used in deep learning, such as Spatial Transformer Networks, rely on bilinear interpolation, which performs poorly under severe scale changes, and more importantly, results in poor gradient propagation. This is due to their strict reliance on direct neighbors. Instead, we propose to generate random auxiliary samples in the vicinity of each pixel in the sampled image, and create a linear approximation with their intensity values. We then use this approximation as a differentiable formula for the transformed image.We demonstrate that our approach produces more representative gradients with a wider basin of convergence for image alignment, which leads to considerable performance improvements when training networks for classification tasks. This is not only true under large downsampling, but also when there are no scale changes. We compare our approach with multi-scale sampling and show that we outperform it. We then demonstrate that our improvements to the sampler are compatible with other tangential improvements to Spatial Transformer Networks and that it further improves their performance. 1 1 Code and models are available at https://github.com/ vcg-uvic/linearized_multisampling_release.

show abstract

Section: Related Workmentioning

confidence: 99%

Linearized Multi-Sampling for Differentiable Image Transformation

Jiang

Sun

Tagliasacchi

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

show abstract

“…This is quite reasonable, since being able to tolerate more patch transformations unavoidably decreases the discrimination power. Concerning OriNet [25], the default patch orientation system of the deep network detailed in [40] seems to be slightly better, possibly due to the difference between the keypoint detector employed during training and that used to generate input patches. RsGLOH2 achieves the best results among the handcrafted descriptors, while RalNet Shuffle and BisGLOH2 ⋆ are comparable with average baseline descriptors.…”

Section: Evaluation Resultsmentioning

confidence: 99%

“…These include SOS-Net [35], still unpublished at contest time, the recent HardNet A [29], obtained by training HardNet [24] on AMOS [29] and other datasets, RalNet Shuffle using the RalNet architecture [38] and additionally cropping and shuffling patches at training time, and RsGLOH2, "square rooting" sGLOH2 [4] according to RootSIFT [1]. Two variants of HardNet A , exploiting the deep networks described in [25] either for custom orientation assignment or to accommodate patches before extracting the descriptor, were also submitted as OriNet+HardNet A and AffNet+HardNet A , respectively. The contest also included a variant of BisGLOH2 [4], named BisGLOH2 ⋆ , using more rotations at matching time than the default ones.…”

Section: Local Image Descriptors Under Evaluationmentioning

confidence: 99%

Which Is Which? Evaluation of Local Descriptors for Image Matching in Real-World Scenarios

Bellavia

Colombo

2019

Computer Analysis of Images and Patterns

View full text Add to dashboard Cite

Matching with local image descriptors is a fundamental task in many computer vision applications. This paper describes the WISW contest held within the framework of the CAIP 2019 conference, aimed at benchmarking recent descriptors in challenging planar and non-planar real image matching scenarios. According to the contest results, the descriptors submitted to the competition, most of which based on deep learning, perform significantly better than the current state-of-the-art in image matching. Nonetheless, there is still room for improvement, especially in the case of non-planar scenes.

show abstract

“…As Eq. 6 indicates that the performance of GMS is related to feature quality, we experiment with recently proposed deep learning based features, including the HardNet (Mishchuk et al 2017) descriptor and HessAff (Mishkin et al 2018) detector. Specifically, we use DoG (Lowe 2004) and Hes-sAff for interest point detection, respectively.…”

Section: Paring With Deep Featuresmentioning

confidence: 99%

GMS: Grid-Based Motion Statistics for Fast, Ultra-robust Feature Correspondence

et al. 2019

View full text Add to dashboard Cite

Feature matching aims at generating correspondences across images, which is widely used in many computer vision tasks. Although considerable progress has been made on feature descriptors and fast matching for initial correspondence hypotheses, selecting good ones from them is still challenging and critical to the overall performance. More importantly, existing methods often take a long computational time, limiting their use in real-time applications. This paper attempts to separate true correspondences from false ones at high speed. We term the proposed method (GMS) grid-based motion Statistics, which incorporates the smoothness constraint into a statistic framework for separation and uses a grid-based implementation for fast calculation. GMS is robust to various challenging image changes, involving in viewpoint, scale, and rotation. It is also fast, e.g., take only 1 or 2 ms in a single CPU thread, even when 50K correspondences are processed. This has important implications for real-time applications. What's more, we show that incorporating GMS into the classic feature matching and epipolar geometry estimation pipeline can significantly boost the overall performance. Finally, we integrate GMS into the well-known ORB-SLAM system for monocular initialization, resulting in a significant improvement. Keywords Feature matching • Epipolar geometry • Visual SLAM • Structure-from-motion • GMS 1 Introduction Feature matching is one of the most fundamental problems in the computer vision community. It aims to generate correspondences across different views of an object or scene, which is widely used in many vision tasks such as structurefrom-motion (Schonberger and Frahm 2016) and Visual SLAM (Davison et al. 2007; Mur-Artal et al. 2015). Typical solutions rely on feature detectors (Harris and stephens 1988), descriptors (Lowe 2004), and matchers (Muja and Communicated by Jiri Matas.

show abstract

Repeatability Is Not Enough: Learning Affine Regions via Discriminability

Cited by 187 publications

References 48 publications

Linearized Multi-Sampling for Differentiable Image Transformation

Linearized Multi-Sampling for Differentiable Image Transformation

Which Is Which? Evaluation of Local Descriptors for Image Matching in Real-World Scenarios

GMS: Grid-Based Motion Statistics for Fast, Ultra-robust Feature Correspondence

Contact Info

Product

Resources

About