Discriminative Learning of Deep Convolutional Feature Point Descriptors

Abstract

Deep learning has revolutionalized image-level tasks such as classification, but patch-level tasks, such as correspondence, still rely on hand-crafted features, e.g. SIFT. In this paper we use Convolutional Neural Networks (CNNs) to learn discriminant patch representations and in particular train a Siamese network with pairs of (non-)corresponding patches. We deal with the large number of potential pairs with the combination of a stochastic sampling of the training set and an aggressive mining strategy biased towards patches that are hard to classify.By using the L 2 distance during both training and testing we develop 128-D descriptors whose euclidean distances reflect patch similarity, and which can be used as a drop-in replacement for any task involving SIFT. We demonstrate consistent performance gains over the state of the art, and generalize well against scaling and rotation, perspective transformation, non-rigid deformation, and illumination changes. Our descriptors are efficient to compute and amenable to modern GPUs, and are publicly available.

FCSS: Fully Convolutional Self-Similarity for Dense Semantic Correspondence

“…Other methods have directly learned a similarity measure for comparing patches using a convolutional similarity network [19,51,41,50]. Even though CNN-based descriptors encode a discriminative structure with a deep architecture, they have inherent limitations in handling large intra-class variations [41,10]. Furthermore, they are mostly tailored to estimate sparse correspondences, and cannot in practice provide dense descriptors due to their high computational complexity.…”

“…Some of these techniques have extracted immediate activations as the descriptor [16,14,9,33], which have shown to be effective for patch-level matching. Other methods have directly learned a similarity measure for comparing patches using a convolutional similarity network [19,51,41,50]. Even though CNN-based descriptors encode a discriminative structure with a deep architecture, they have inherent limitations in handling large intra-class variations [41,10].…”

“…Over the past few years, convolutional neural network (CNN) based features have become increasingly popular for correspondence estimation thanks to their localization precision of matched points and their invariance to minor geometric deformations and illumination changes [19,51,41,50]. However, for computing semantic correspondences within this framework, greater invariance is needed to deal with the more substantial appearance differences.…”

Learning Deep Relationship for Object Detection

“…To illustrate the advantage of fine tuning, a Siamese-like approach DeepDesc [14] is used to learn discriminative features, and kNN is utilized to classify the features.…”

Accurate and Linear Time Pose Estimation from Points and Lines

“…Early approaches were focused on solving the problem for the minimal cases with n = {3, 4, 5} [7,11,12,15,19,42]. The proliferation of feature point detectors [16,36] and descriptors [3,26,29,37,40] able to consistently retrieve many feature points per image, brought a series of new PnP algorithms that could efficiently handle arbitrarily large sets of points [9,13,18,24,25,27,30,38,45]. Amongst…”