R2D2: Repeatable and Reliable Detector and Descriptor

Abstract

Interest point detection and local feature description are fundamental steps in many computer vision applications. Classical methods for these tasks are based on a detect-thendescribe paradigm where separate handcrafted methods are used to first identify repeatable keypoints and then represent them with a local descriptor. Neural networks trained with metric learning losses have recently caught up with these techniques, focusing on learning repeatable saliency maps for keypoint detection and learning descriptors at the detected keypoint locations. In this work, we argue that salient regions are not necessarily discriminative, and therefore can harm the performance of the description. Furthermore, we claim that descriptors should be learned only in regions for which matching can be performed with high confidence. We thus propose to jointly learn keypoint detection and description together with a predictor of the local descriptor discriminativeness. This allows us to avoid ambiguous areas and leads to reliable keypoint detections and descriptions. Our detection-and-description approach, trained with selfsupervision, can simultaneously output sparse, repeatable and reliable keypoints that outperforms state-of-the-art detectors and descriptors on the HPatches dataset. It also establishes a record on the recently released Aachen Day-Night localization benchmark.

Robust Partial-to-Partial Point Cloud Registration in a Full Range

“…Repetitive and Symmetric Structures. Existing research works, such as R2D2 [34] and SuperGlue [40], consider encoding high-quality features for matching similar image features. Similar strategies could also facilitate PPR for the following reasons: 1) not all points are equally important 3D features [46]; 2) man-made objects often have many repetitive and symmetric 3D structures.…”

A Critical Analysis of Image-based Camera Pose Estimation Techniques

“…They jointly trained a network called SuperPoint for interest point detection and description. Other than only learning key points, a descriptor R2D2 [77] is used to train a predictor of the local descriptor discriminator. They argued that salient but discriminative regions can harm performance.…”

Review of Wide-Baseline Stereo Image Matching Based on Deep Learning

“…The encoding structure is used for image feature extraction, whereas the decoding structure can not only output the position of the feature point, but also output the descriptor vector. Similarly, Revaud et al [84] proposed the Siamese decoding structure R2D2, which focuses more on the repetitive and discriminative expression of training features than SuperPoint.…”