D-Nets: Beyond patch-based image descriptors

Abstract

Despite much research on patch-based descriptors, SIFT remains the gold standard for finding correspondences across images and recent descriptors focus primarily on improving speed rather than accuracy. In this paper we propose Descriptor-Nets (D-Nets), a computationally efficient method that significantly improves the accuracy of image matching by going beyond patch-based approaches. D-Nets constructs a network in which nodes correspond to traditional sparsely or densely sampled keypoints, and where image content is sampled from selected edges in this net. Not only is our proposed representation invariant to cropping, translation, scale, reflection and rotation, but it is also significantly more robust to severe perspective and non-linear distortions. We present several variants of our algorithm, including one that tunes itself to the image complexity and an efficient parallelized variant that employs a fixed grid. Comprehensive direct comparisons against SIFT and ORB on standard datasets demonstrate that D-Nets dominates existing approaches in terms of precision and recall while retaining computational efficiency.

Ultra-wide Baseline Aerial Imagery Matching in Urban Environments

“…We believe that A-SIFT encapsulates SIFT by definition, and therefore we do not compare with the standard SIFT. For D-Nets [33], we use the implementation provided on their website in a straight forward manner employing the FAST keypoint detector. We measure whether each of these methods find the correct homography, or finds a shifted version of the correct one, or finds a correct but different plane, or completely fails.…”

“…This leads to a heavy dependence on the matching and robust estimation approach, because random sampling cannot be prevented from mixing different affine transformations in a local region. D-Nets [33] take a different approach in finding correspondences. Their method generates lines between keypoints or grid points and calculates descriptors for each line.…”

A Spherical Model Based Keypoint Descriptor and Matching Algorithm for Omnidirectional Images

“…Although the pixel histogram descriptor is effective in food recognition, it is not necessarily suitable for finding correspondences between omnidirectional images. By using the keypoint pair, we present a new descriptor here based on the Descriptor-Nets (D-Nets) [15] for the matching of omnidirectional images. The keypoint pair is sampled from the image curve in the image plane, and so the descriptor contains the feature information of nonlinear distortion and the image matching is robust.…”

Scene Signatures: Localised and Point-less Features for Localisation

“…Our scene-signature detection block runs in a separate thread at approximately 2-5 Hz. Our main thread runs Visual Odometry at approximately [15][16][17][18][19][20] Hz to predict poses in between localisations. As the localisation updates occur at a slower rate, we perform posegraph relaxation over a sliding window to obtain our final estimate (see Figure 10).…”

“…However, as this is still based on low-level structure, data association remains hard under extreme appearance change. Hundelshausen et al [16] present a noteworthy descriptor that goes beyond point features and instead constructs a network of nodes and directed edges, where each edge is a descriptor in the network, referred to as a "d-token". However, because these descriptors directly sample pixel intensities, this would not be suitable for the types of extreme appearance changes we are considering.…”