Revisiting Local Descriptor Based Image-To-Class Measure for Few-Shot Learning

Abstract

Few-shot learning in image classification aims to learn a classifier to classify images when only few training examples are available for each class. Recent work has achieved promising classification performance, where an image-level feature based measure is usually used. In this paper, we argue that a measure at such a level may not be effective enough in light of the scarcity of examples in few-shot learning. Instead, we think a local descriptor based image-to-class measure should be taken, inspired by its surprising success in the heydays of local invariant features. Specifically, building upon the recent episodic training mechanism, we propose a Deep Nearest Neighbor Neural Network (DN4 in short) and train it in an end-to-end manner. Its key difference from the literature is the replacement of the image-level feature based measure in the final layer by a local descriptor based image-to-class measure. This measure is conducted online via a k-nearest neighbor search over the deep local descriptors of convolutional feature maps. The proposed DN4 not only learns the optimal deep local descriptors for the image-to-class measure, but also utilizes the higher efficiency of such a measure in the case of example scarcity, thanks to the exchangeability of visual patterns across the images in the same class. Our work leads to a simple, effective, and computationally efficient framework for few-shot learning. Experimental study on benchmark datasets consistently shows its superiority over the related stateof-the-art, with the largest absolute improvement of 17% over the next best. The source code can be available from https://github.com/WenbinLee/DN4.git.

Compare More Nuanced: Pairwise Alignment Bilinear Network for Few-Shot Fine-Grained Learning

“…Finally, we analyze the experimental results of the proposed models and compare them with other few-shot learning approaches. For a fair comparison, we conduct two groups of experiments on these datasets, for the first group, we follow the setting, which Wei et al [1], [23] used, while for the second group, we follow the newest settings in the recent few-shot methods [19], [20].…”

“…Another class of few-shot learning methods follows the idea of learning to compare [15]- [17], [20], [55]. In general, these approaches consist of two main components: a feature embedding network and a similarity metric.…”

“…These methods aim to optimize the transferable embedding of both auxiliary data and target data. Consequently, the test images can to be identified by the simple nearest neighbor classifier [15], [16], deep distance matrix based classifier [17] or cosine distance based classifier [20], [55]. Considering the FSFG task requires the more advanced information processing ability, we propose to capture more nuanced features from the images pairs other than the first-order extraction used in leaning to compare approaches.…”

“…It uses a mini network to learn the similarity between the query image and the support class. • DN4 [20], the newest generic few-shot method published in CVPR 2019. By using a deep nearest neighbor neural netwok, DN4 can aggregate the discriminative [19], the newest FSFG model published in AAAI 2019.…”

“…ability, machine few-shot learning is initially identified by Li et al [14], they propose to utilize probabilistic models to represent object categories and update them with few training examples. Most recently, inspired by the advanced representation learning ability of deep neural networks, deep machine few-shot learning [15]- [20] revives and achieves significant improvements against previous methods. However, considering the cognitive process of human beings, preschool students can easily distinguish the difference between generic concepts like the 'Cat' and 'Dog' after seeing a few exemplary images of these animals, but they may be confused about finegrained dog categories such as the 'Husky' and 'Alaskan' with limited samples.…”