ALIKE: Accurate and Lightweight Keypoint Detection and Descriptor Extraction

Zhao, Xiaoming; Wu, Xiaofeng; Miao, Jinyu; Chen, Weihai; Chen, Peter C. Y.; Li, Zhengguo

doi:10.1109/tmm.2022.3155927

Cited by 38 publications

(32 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Concerning deep image matching methods, we included the current state-of-the-art according to recent evaluation (Remondino et al, 2021;Chen et al, 2021;Jin et al, 2021;Ma et al, 2021;Bellavia et al, 2022b): the DIScrete Keypoints (DISK, Tyszkiewicz et al, 2020), the Hybrid Pipeline (HP, Bellavia et al, 2022c) also without rotational invariance provided by OriNet (Mishkin et al, 2018) (denoted as HP_upright ), the Local Feature TRansformer (LoFTR, Sun et al, 2021) and its rotation invariant extension SE2-LoFTR (Bökman et al, 2022), SuperPoint+SuperGlue (Sarlin et al, 2020), the Accurate Shape and Localization Features (ASLFeat, Luo et al, 2020), the Repeatable Detector and Descriptor (R2D2, Revaud et al, 2019) and the Local Feature Network (LF-Net, Ono et al, 2018). Two further deep-learning methods providing quite interesting results were also added in this evaluation: the Rotation-Robust Descriptors (RoRD, Parihar et al, 2021) for its rotation invariance and the Accurate and Lightweight Keypoint Detection and Descriptor Extraction (ALIKE, Zhao et al, 2022) for its ability to run in real-time.…”

Section: Compared Methodsmentioning

confidence: 99%

Photogrammetry Now and Then – From Hand-Crafted to Deep-Learning Tie Points –

Morelli

Bellavia

Menna

et al. 2022

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

Abstract. Historical images provide a valuable source of information exploited by several kinds of applications, such as the monitoring of cities and territories, the reconstruction of destroyed buildings, and are increasingly being shared for cultural promotion projects through virtual reality or augmented reality applications. Finding reliable and accurate matches between historical and present images is a fundamental step for such tasks since they require to co-register the present 3D scene with the past one. Classical image matching solutions are sensitive to strong radiometric variations within the images, which are particularly relevant in these multi-temporal contexts due to different types of sensitive media (film/sensors) employed for the image acquisitions, different lighting conditions and viewpoint angles. In this work, we investigate the actual improvement provided by recent deep learning approaches to match historical and nowadays images. As learning-based methods have been trained to find reliable matches in challenging scenarios, including large viewpoint and illumination changes, they could overcome the limitations of classic hand-crafted methods such as SIFT and ORB. The most relevant approaches proposed by the research community in the last years are analyzed and compared using pairs of multi-temporal images.

show abstract

Section: Compared Methodsmentioning

confidence: 99%

Photogrammetry Now and Then – From Hand-Crafted to Deep-Learning Tie Points –

Morelli

Bellavia

Menna

et al. 2022

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

show abstract

“…DISK [22] uses reinforcement learning to train the score map and descriptor map. ALIKE [10] has a differentiable keypoint detection module for accurate keypoint training and has the lightest network, thereby allowing its application in real-time visual measurement applications. D2-Net [23] does not estimate the score map with the network, but rather detects keypoints with II).…”

Section: B Joint Keypoint and Descriptor Learningmentioning

confidence: 99%

“…Initially, DNNs were used to extract descriptors of image patches at predefined keypoints [7]. Subsequently, the mainstream approach became the extraction of keypoints and descriptors with a single network [8]- [10], which can often extract more robust keypoints and discriminative descriptors than hand-crafted methods [11]. We refer to these methods as map-based methods because they estimate a score map and a descriptor map using two heads: the score map head (SMH) and the descriptor map head (DMH).…”

Section: Introductionmentioning

confidence: 99%

“…Afterward, we propose A LIghter Keypoint and descriptor Extraction network with Deformable transformation (ALIKED) for visual measurement using the SDDH. However, the SDDH extracts only sparse descriptors, which means there are no descriptor maps for constructing neural reprojection error (NRE) loss [10], [17]. To address this challenge, we propose an elegant solution to relax the NRE loss from dense to sparse.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

ALIKED: A Lighter Keypoint and Descriptor Extraction Network via Deformable Transformation

Zhao

Chen

et al. 2023

IEEE Trans. Instrum. Meas.

View full text Add to dashboard Cite

Image keypoints and descriptors play a crucial role in many visual measurement tasks. In recent years, deep neural networks have been widely used to improve the performance of keypoint and descriptor extraction. However, the conventional convolution operations do not provide the geometric invariance required for the descriptor. To address this issue, we propose the Sparse Deformable Descriptor Head (SDDH), which learns the deformable positions of supporting features for each keypoint and constructs deformable descriptors. Furthermore, SDDH extracts descriptors at sparse keypoints instead of a dense descriptor map, which enables efficient extraction of descriptors with strong expressiveness. In addition, we relax the neural reprojection error (NRE) loss from dense to sparse to train the extracted sparse descriptors. Experimental results show that the proposed network is both efficient and powerful in various visual measurement tasks, including image matching, 3D reconstruction, and visual relocalization.

show abstract

“…This step is non-differentiable, therefore we apply a differentiable subpixel keypoint refinement (DKR) that allows the gradients to flow back to the small regions around the keypoints. Inspired by recent works [14,15,11,27], we extract 5 × 5 patches from the confidence heatmap which are centered at the N max keypoint positions and compute for each patch p the spatial softargmax of the normalized patch (p − s NMS )/t, where s NMS is the value of the NMS score map and t the temperature for the softmax. The refined keypoint positions are the sum of the initial coordinates and the soft subpixel coordinates.…”

Section: Multi-modal Retinal Keypoint Detection and Description Networkmentioning

confidence: 99%

Multi-modal Retinal Image Registration Using a Keypoint-Based Vessel Structure Aligning Network

Sindel¹,

Hohberger²,

Maier³

et al. 2022

Preprint

View full text Add to dashboard Cite

In ophthalmological imaging, multiple imaging systems, such as color fundus, infrared, fluorescein angiography, optical coherence tomography (OCT) or OCT angiography, are often involved to make a diagnosis of retinal disease. Multi-modal retinal registration techniques can assist ophthalmologists by providing a pixel-based comparison of aligned vessel structures in images from different modalities or acquisition times. To this end, we propose an end-to-end trainable deep learning method for multi-modal retinal image registration. Our method extracts convolutional features from the vessel structure for keypoint detection and description and uses a graph neural network for feature matching. The keypoint detection and description network and graph neural network are jointly trained in a self-supervised manner using synthetic multi-modal image pairs and are guided by synthetically sampled ground truth homographies. Our method demonstrates higher registration accuracy as competing methods for our synthetic retinal dataset and generalizes well for our real macula dataset and a public fundus dataset.

show abstract

ALIKE: Accurate and Lightweight Keypoint Detection and Descriptor Extraction

Cited by 38 publications

References 40 publications

Photogrammetry Now and Then – From Hand-Crafted to Deep-Learning Tie Points –

Photogrammetry Now and Then – From Hand-Crafted to Deep-Learning Tie Points –

ALIKED: A Lighter Keypoint and Descriptor Extraction Network via Deformable Transformation

Multi-modal Retinal Image Registration Using a Keypoint-Based Vessel Structure Aligning Network

Contact Info

Product

Resources

About